Kirkpatrick's Four-Level Training Evaluation Model

Analyzing Training Effectiveness

Kirkpatrick's Four-Level Training Evaluation Model

Evaluate training effectively.

© iStockphoto/Jeryco

If you deliver training for your team or your organization, then you probably know how important it is to measure its effectiveness. After all, you don't want to spend time or money on training that doesn't provide a good return.

This is where Kirkpatrick's Four-Level Training Evaluation Model can help you objectively analyze the effectiveness and impact of your training, so that you can improve it in the future.

In this article, we'll look at each of the Kirkpatrick four levels, and we'll examine how you can apply the model to evaluate training. We'll also look at some of the situations where the model may not be useful.

The Four Levels

Donald Kirkpatrick, Professor Emeritus at the University of Wisconsin and past president of the American Society for Training and Development (ASTD), first published his Four-Level Training Evaluation Model in 1959, in the US Training and Development Journal.

The model was then updated in 1975, and again in 1994, when he published his best-known work, "Evaluating Training Programs."

The four levels are:

  1. Reaction.
  2. Learning.
  3. Behavior.
  4. Results.

Let's look at each level in greater detail.

Level 1: Reaction

This level measures how your trainees (the people being trained), reacted to the training. Obviously, you want them to feel that the training was a valuable experience, and you want them to feel good about the instructor, the topic, the material, its presentation, and the venue.

It's important to measure reaction, because it helps you understand how well the training was received by your audience. It also helps you improve the training for future trainees, including identifying important areas or topics that are missing from the training.

Level 2: Learning

At level 2, you measure what your trainees have learned. How much has their knowledge increased as a result of the training?

When you planned the training session, you hopefully started with a list of specific learning objectives: these should be the starting point for your measurement. Keep in mind that you can measure learning in different ways depending on these objectives, and depending on whether you're interested in changes to knowledge, skills, or attitude.

It's important to measure this, because knowing what your trainees are learning and what they aren't will help you improve future training.

Level 3: Behavior

At this level, you evaluate how far your trainees have changed their behavior, based on the training they received. Specifically, this looks at how trainees apply the information.

It's important to realize that behavior can only change if conditions are favorable. For instance, imagine you've skipped measurement at the first two Kirkpatrick levels and, when looking at your group's behavior, you determine that no behavior change has taken place. Therefore, you assume that your trainees haven't learned anything and that the training was ineffective.

However, just because behavior hasn't changed, it doesn't mean that trainees haven't learned anything. Perhaps their boss won't let them apply new knowledge. Or, maybe they've learned everything you taught, but they have no desire to apply the knowledge themselves.

Level 4: Results

At this level, you analyze the final results of your training. This includes outcomes that you or your organization have determined to be good for business, good for the employees, or good for the bottom line.

Tip:

Make sure that you plan your training effectively. Use our articles on Training Needs Assessment  , Gagne's Nine Levels of Learning   and 4MAT   to help you do this.

How to Apply the Model

Level 1: Reaction

Start by identifying how you'll measure reaction. Consider addressing these questions:

  • Did the trainees feel that the training was worth their time?
  • Did they think that it was successful?
  • What were the biggest strengths of the training, and the biggest weaknesses?
  • Did they like the venue and presentation style?
  • Did the training session accommodate their personal learning styles  ?

Next, identify how you want to measure these reactions. To do this you'll typically use employee satisfaction surveys   or questionnaires; however you can also watch trainees' body language during the training, and get verbal feedback by asking trainees directly about their experience.

Once you've gathered this information, look at it carefully. Then, think about what changes you could make, based on your trainees' feedback and suggestions.

Level 2: Learning

To measure learning, start by identifying what you want to evaluate. (These things could be changes in knowledge, skills, or attitudes.)

It's often helpful to measure these areas both before and after training. So, before training commences, test your trainees to determine their knowledge, skill levels, and attitudes.

Once training is finished, test your trainees a second time to measure what they have learned, or measure learning with interviews or verbal assessments.

Level 3: Behavior

It can be challenging to measure behavior effectively. This is a longer-term activity that should take place weeks or months after the initial training.

Consider these questions:

  • Did the trainees put any of their learning to use?
  • Are trainees able to teach their new knowledge, skills, or attitudes to other people?
  • Are trainees aware that they've changed their behavior?

One of the best ways to measure behavior is to conduct observations and interviews over time.

Also, keep in mind that behavior will only change if conditions are favorable. For instance, effective learning could have taken place in the training session. But, if the overall organizational culture isn't set up for any behavior changes, the trainees might not be able to apply what they've learned.

Alternatively, trainees might not receive support, recognition, or reward for their behavior change from their boss. So, over time, they disregard the skills or knowledge that they have learned, and go back to their old behaviors.

Level 4: Results

Of all the levels, measuring the final results of the training is likely to be the most costly and time consuming. The biggest challenges are identifying which outcomes, benefits, or final results are most closely linked to the training, and coming up with an effective way to measure these outcomes over the long term.

Here are some outcomes to consider, depending on the objectives of your training:

  • Increased employee retention.
  • Increased production.
  • Higher morale.
  • Reduced waste.
  • Increased sales.
  • Higher quality ratings.
  • Increased customer satisfaction.
  • Fewer staff complaints.

Considerations

Although Kirkpatrick's Four-Level Training Evaluation Model is popular and widely used, there are a number of considerations that need to be taken into account when using the model.

One issue is that it can be time-consuming and expensive to use levels 3 or 4 of the model, so it's not practical for all organizations and situations. This is especially the case for organizations that don't have a dedicated training or human resource department, or for one-off training sessions or programs.

In a similar way, it can be expensive and resource intensive to "wire up an organization" to collect data with the sole purpose of evaluating training at levels 3 and 4. (Whether or not this is practical depends on the systems already in place within the organization.)

The model also assumes that each level's importance is greater than the last level, and that all levels are linked. For instance, it implies that Reaction is less important, ultimately, than Results, and that reactions must be positive for learning to take place. In practice, this may not be the case.

Most importantly, organizations change in many ways, and behaviors and results change depending on these, as well as on training. For example, measurable improvements in areas like retention and productivity could result from the arrival of a new boss or from a new computer system, rather than from training.

Kirkpatrick's model is great for trying to evaluate training in a "scientific" way, however, so many variables can be changing in fast-changing organizations that analysis at level 4 can be limited in usefulness.

Key Points

The Kirkpatrick Four-Level Training Evaluation Model helps trainers to measure the effectiveness of their training in an objective way. The model was originally created by Donald Kirkpatrick in 1959, and has since gone through several updates and revisions.

The Four-Levels are as follows:

  1. Reaction.
  2. Learning.
  3. Behavior.
  4. Results.

By going through and analyzing each of these four levels, you can gain a thorough understanding of how effective your training was, and how you can improve in the future.

Bear in mind that the model isn't practical in all situations, and that measuring the effectiveness of training with the model can be time-consuming and use a lot of resources.

This site teaches you the skills you need for a happy and successful career; and this is just one of many tools and resources that you'll find here at Mind Tools. Click here for more, subscribe to our free newsletter, or become a member for just $1.


Comments (6)
  • Midgie wrote Over a month ago
    Hi GB,
    Indeed it does sound like a fun project to be involved with! Good luck with it all and remember we're here if you want to bounce around ideas.

    And thanks for that feedback from what you've seen on my LinkedIn updates. My business strap line is 'Inspiring Excellence' so being a role model and demonstrating some of that excellence is important to me.

    Midgie
  • GoldenBoy wrote Over a month ago
    We've been going through some significant changes recently (for us, it seems that change is the only constant LOL).

    Part of this has involved professional development to look at best practices in service delivery models, not-for-profit employment services sector.

    We need to develop some measurables for the variation in client successes, despite seemingly similar opportunities. Also, our service model is quite linear, and this looks like it needs to change. We are looking to Level Three and Level Four to evaluate our training delivery, and what we can keep and what needs to change.

    This is also something we plan to implement at the beginning of any training modifications, reverse engineering the training decision to better meet client needs from bot the supply and demand side of our business.

    As to success criteria, that's not an easy one to answer. There are several variables, such as is a client successful just because they got A job, or only when they find an opportunity in their professional field? Is success measured by the number of employers who hire placements? Is any successful hire a success, or only if their opportunity becomes a sustainable one? And so on...

    We'll iron out the kinks over the next couple of months I'm sure, and it'll be a fun project to be involved in.

    BTW Midgie, I've been following you on LinkedIn - you're an awesome role model. Keep up the great work!

    TTFN
    GB
  • Midgie wrote Over a month ago
    Hey GB,
    Good to hear from you. Seems like ages since we last saw you around!

    I'm curious, what has prompted you to now evaluate your training? Has there been any issues or concerns, or do you simply have the space to gaze out to the horizon and explore things?

    If you are measuring success, what will your success criteria be?

    It would be great to hear your thoughts, as we can all learn and remind ourselves of what it takes to make and deliver a successful training course!

    Midgie
  • GoldenBoy wrote Over a month ago
    I realise the article is older now, but the information is still invaluable. I've just started taking a look into measuring the success of our training at Level 3 and Level 4, which should have been done years ago. Nonetheless, I'm intrigued by this approach and its sustainability despite the age of the theory.

    TTFN,
    GB
  • MichaelP wrote Over a month ago
    Michele, tks for sharing your experiences and this great advice.

    I am going to be better prepared when I next have to plan a training program.

    cheers
  • zuni wrote Over a month ago
    I have used Kirkpatrick's 4-level evaluation model extensively in my work in the learning and development function. I cannot stress enough how important it is to define your learning evaluation approach as part of the learning design. Too often the focus is on developing and delivering the content with learning evaluation becoming a focus only after the learning program is launched. Attempting to develop the evaluation as an afterthought can be difficult and the integrity of the results are often compromised.

    When designing training, consider the following sequence:

    - Business direction
    - Learning needs assessment to identify issues and learning gaps
    - Develop learning objectives (including anticipated results)
    - Design learning evaluation approach
    - Develop learning design
    - Develop content
    - Deliver a pilot training session to a representative group of learners (if it is a large training initiative) to test content, delivery approach and evaluation of the learning)
    - Launch training and collect data

    Michele
    -

Where to go from here:

Join the Mind Tools Club

Click to join Mind Tools
Printer-friendly version
Return to the top of the page

Create a Login to Save Your Learning Plan

This ensures that you don’t lose your plan.


Connect with…

Or create a Mind Tools login. Existing user? Log in here.
Log in with your existing Mind Tools details
Lost Username or Password
You are now logged in…

Lost username or password?

Please enter your username or email address and we'll send you a reminder.

Thank You!

Your log in details have been sent to the email account you registered with. Please check your email to reset your login details.

Create a Mind Tools Login
Your plan has been created.

While you're here, subscribe to our FREE newsletter?

Learn a new career skill every week, and get our Personal Development Plan workbook (worth $19.99) when you subscribe.


Thank You!

Please check your Inbox, and click on the link in the email from us. We can then send you the newsletter.