Monitoring and Evaluating the Outcomes of Change

Organizational performance is heavily dependent on, and influenced by, the level of engagement of its staff. Measuring the effectiveness of the HR function can often reveal issues that impact directly on the bottom line. Here, engagement and employee satisfaction measures are used:

• rate of staff turnover and associated recruitment costs
• absenteeism – long- and short-term sickness
• average length of service
• number of disciplinary and grievance cases
• results of employee opinion/attitude surveys

The analysis of the information gleaned from the measures would not only reveal the current level of engagement, but would also help set targets, objectives and budgets for succession planning, training and development initiatives and recruitment.

2. Productivity

It should be common practice for organizations to regularly examine their systems of efficiency around the creation of outputs and change or amend them accordingly. Productivity measures center on:

• the time, costs and resources needed to design, develop, create and deliver a product or service
• the quantity of outputs, or number of customers served, etc.
• the amount of variation in the quality of the products or services, i.e. the number of errors or defects

These types of measures are easily quantifiable and can be obtained from production, financial and sales reports. They will be statistical in nature and many organizations will have automated systems, particularly on production lines and in call centers, for recording the data.

3. Employee Performance

An organization may instigate a formal change program to improve the performance of its employees, or it may adopt a continuous improvement approach to increasing the skills and expertise of its staff. Improvement initiatives will center on the development of competencies:

• skills
• knowledge
• experience
• behaviors

Initiatives might include formal training and development courses, coaching, mentoring, secondments, action learning sets, self-directed learning programs, on the job learning etc. Measuring the change in, and development of, personal competence is important as development initiatives can often be costly and protracted (some programs take 12 months or more to complete). Competency measures should focus on finding answers to the following questions:

• Are individuals or groups demonstrating and applying higher levels of skill and knowledge?
• Has the individual or group’s outputs improved in terms of quality and quantity of work? For example, have sales increased following a sales techniques training course?
• Has an individual's or group’s behavior changed in favor of organizational standards and values? For example, are people more committed to customer service? Are interpersonal and interdepartmental relationships more effective? Is there a focus on professionalism – time-keeping, personal presentation, loyalty etc?

Gathering the data to measure competencies can be very difficult and often depends on the subjective appraisal of other people. Methods for gathering assessment information might include:

• evidence of employees attending training and applying the learning, e.g. by using a new computer program
• evidence of employees gaining formal qualifications, e.g. NVQs, diplomas, certificates, degrees, etc.
• formal testing, e.g. a driving test
• psychometric tests that determine personality strengths and behavioral preferences
• self-assessment, where individuals rate their own level of learning and indicate how they feel their performance has improved
• assessment centers, where individuals are asked to perform a number of tasks that are designed to demonstrate levels of knowledge, skill and ability, e.g. making a presentation, discussing a certain topic, answering a case study exercise, etc.
• observation/feedback from others using questionnaires, discussions and reports, for example:
• observing individuals or groups performing in the work place
• 360-degree feedback
• stakeholder/customer feedback questionnaires
• review of performance against annual objectives by line managers
• focus groups, where a number of people assess the performance of individuals or groups

A combination of the above techniques might be necessary to measure levels of performance accurately. It might also prove useful to measure the performance levels of an individual or group that has undergone certain training and compare it to a control group that has not.

4. Customer Service

A change program to improve the service received by customers, and other stakeholders, can be measured quantitatively by monitoring:

• number of customers
• value (monetary) of the average customer – i.e. average spend per customer
• number of complaints or goods returned
• level of loyalty – number of return visits, repeat business, length of membership, etc.
• number of transactions, i.e. customers served, calls received, orders taken, enquiries dealt with, etc.

Customer satisfaction surveys can also provide useful qualitative information that can be used to determine whether customers feel that the service has improved or declined over time.

5. Technology

The success of a change program that introduces new technology in order to improve performance can be measured by analyzing the effectiveness of processes prior to the technology implementation and afterwards. Measures will usually be in terms of improvements in:

• time
• cost
• productivity
• quality

Employee opinion surveys are also a very useful measure of what has improved or not, as the case may be. Questionnaires designed to determine ease of operation, reliability etc. will give a sense of the indirect impact of such a change initiative. New technology systems can often have an adverse effect on other functions or processes that previously appeared unrelated. For example, has the technology negated the need for people to meet and interact, resulting in the deterioration of working relationships?

6. Sales and Marketing

Effective marketing activities should result in increased sales. They can also manifest themselves in an increase in brand awareness. Measures of the effectiveness of sales and marketing activities could include:

• sales revenue and sales costs
• increase in market share (percentage of the total market sales the organization is able to capture)
• level of brand awareness determined by market research, e.g. website hits, mail surveys, door-to-door interviews, telephone interviews and benchmarking against other brands
• number of enquiries about goods or services from customers or potential customers received by phone, email, letter etc.
• number of enquiries received by the organization regarding employment opportunities

Summary

It’s imperative to know whether a change initiative has had the desired impact or not. Therefore, it is very important to determine at the outset of the project what outcomes are expected, how they will be measured and when.

Dependent on the initiative, measures will either focus on the performance of functions, processes or people. There are numerous ways in which to measure performance outcomes using data, figures or subjective information. Whichever method is used it is, of course, important to measure levels of performance prior to any project then compare them again afterwards. Remember that some initiatives will be monitored over a period of time, while others will be designed to create immediate improvements.