The Kirkpatrick Model of Training Evaluation
Turn this article into takeaways for your work.
Each assistant summarizes the article only for you and suggests best practices for your work.
Ask a room of learning professionals how they evaluate training and a large share will say "Kirkpatrick." The four-level model is the default vocabulary of the field. It shows up in vendor dashboards, in RFPs, in certification syllabi and in the first slide of almost every measurement workshop.
Yet it's often used loosely. People quote "Level 4" without defining it, or treat the levels as a ladder you climb after the course ends. This article is a reference on the model itself: where it came from, what each level means and how to measure it, how the New World update changed the way it's meant to be used, what the research literature found wrong with it, and how to apply it without fooling yourself. If your interest is the commercial angle, the sibling piece on proving training ROI to buyers covers how sellers build a business case around these ideas.
Key Facts
- The model has four levels: Reaction, Learning, Behavior and Results. Kirkpatrick Partners credits Dr. Donald Kirkpatrick with creating the four-part structure in the 1950s (Kirkpatrick Partners).
- Kirkpatrick set out the approach in four articles in the journal of the American Society of Training Directors, starting in November 1959 and ending in February 1960 (Nickols).
- The update known as the New World Kirkpatrick Model asks evaluators to start with Level 4 and work backward, and adds required drivers and leading indicators (Kirkpatrick Partners).
- In 1989, Alliger and Janak named three problematic assumptions in the model: that the levels ascend in information value, that they are causally linked and that they are positively intercorrelated (Alliger and Janak, 1989).
- Holton argued in 1996 that the four-level system is really a taxonomy of outcomes and is flawed as an evaluation model (Holton, 1996).
Where the model came from
Donald Kirkpatrick set out his four-level approach in a series of four articles in the journal of what was then the American Society of Training Directors. The first appeared in November 1959 and the last in February 1960. The articles were later collected in a 1975 compilation, Evaluating Training Programs, published by ASTD (Nickols). Kirkpatrick Partners adds that he established the framework around the time he completed his doctoral dissertation, Evaluating a Human Relations Training Program for Supervisors (Kirkpatrick Partners).
The model is old, written for a profession of training directors running mostly instructor-led programs. And the original levels were thinly defined. In his original wording, reaction was "how well the trainees liked a particular training program," and for Results he offered examples rather than a definition: reduced costs, reduced turnover and absenteeism, fewer grievances, higher quality and quantity of production, or improved morale (Nickols). Much of what people now treat as "the Kirkpatrick Model" was added by later writers, most visibly Donald's son James and his wife Wendy, whose updates Kirkpatrick Partners describes as emerging in the 2010s (Kirkpatrick Partners). The third edition of the book, written with James, is still published by Berrett-Koehler (Penguin).
The four levels in depth
Kirkpatrick Partners currently defines the levels like this (source):
| Level | Name | Definition (current wording, abridged) |
|---|---|---|
| 1 | Reaction | Whether participants find the experience favorable, engaging and relevant to the work |
| 2 | Learning | The degree to which people acquire the intended knowledge, skills, attitude, confidence and commitment |
| 3 | Behavior | Whether people perform the critical behaviors in their work environment, supported by it and accountable for it |
| 4 | Results | The degree to which targeted organizational outcomes occur as a result of the experience or initiative |
The numbered order is the order of data collection in a traditional evaluation. As you'll see below, it is not the order the New World version tells you to plan in.
Level 1: Reaction
Level 1 asks how participants experienced the program. The original framing was about whether they liked it. The current definition adds engagement and relevance to their job, which matters because "I enjoyed the facilitator" and "I can use this on Monday" are very different answers.
Example measures:
- A post-session survey with items on relevance, as well as enjoyment
- Net promoter style "would you recommend this" questions
- Facilitator observation of engagement during the session
- In digital programs, drop-off points and optional-activity uptake (see learner engagement metrics)
Level 1 is cheap, fast and useful as a diagnostic for the program itself. Its limit is that it measures the experience, not the outcome. A smooth, entertaining course can teach nothing, and a demanding one can teach a great deal.
Level 2: Learning
Level 2 asks whether people acquired what the program intended. Kirkpatrick's original question was what principles, facts and techniques were understood and absorbed. Nickols points out that to say learning resulted from the training, you need to know the learner's starting point, which means measuring before as well as after (Nickols).
Example measures:
- Pre-test and post-test scores on the same instrument
- A practical skills demonstration or simulation, scored with a rubric
- Self-rated confidence before and after, which maps to the "confidence" part of the current definition
- A written statement of commitment to apply specific actions, which maps to "commitment"
Level 2 is where most course platforms stop, because quiz data is automatic. It tells you what a person could do at the end of the program. It says nothing about whether they will do it back at work.
Level 3: Behavior
Level 3 asks whether people do things differently on the job. This is the level most programs lose, because learning has to survive contact with a workplace that may not support it. The current definition builds that into the wording: people perform the critical behaviors and are supported in and accountable for them.
Example measures:
- Manager or peer observation using a checklist of the specific behaviors
- Review of work products, such as a sample of call recordings, code reviews or customer emails
- Follow-up surveys of participants and their managers a few weeks or months later
- System data that reflects the behavior, such as use of a new process or tool
The measurement challenge is timing and ownership. Behavior change takes weeks to show, and the person best placed to see it is usually the participant's manager, not the L&D team. Getting a manager to complete an observation is a design problem as much as a measurement one. For the research on why learning does or doesn't carry over, see learning transfer.
Level 4: Results
Level 4 asks whether the targeted organizational outcomes occurred. Note the word "targeted." The model doesn't ask you to find any business metric that moved. It asks about the outcomes you and your stakeholders said you were after.
Example measures:
- Revenue, win rate or deal size for a sales program
- Error rates, defects or safety incidents for an operations program
- Customer satisfaction scores or complaint volume for a service program
- Retention, time to productivity or absenteeism for a people-management program
Level 4 is the hardest level for a simple reason: many things move a business metric at once. A training program rarely acts alone. That's where evaluators reach for comparison groups, trend baselines or stakeholder estimates, which the proving training ROI article walks through.
The New World Kirkpatrick Model
The most important change in the modern version is a change of sequence. The traditional levels read like a timeline: run the course, then ask four questions. The New World version turns that around. Kirkpatrick Partners describes it as emphasizing that you start with Level 4 and work backward through design (Kirkpatrick Partners).
A working order looks like this:
- Define the Level 4 result. What organizational outcome should change, and who cares?
- Identify leading indicators. These are short-term observations and measurements that suggest critical behaviors are on track to create a positive impact on the desired results (Kirkpatrick Partners workbook).
- Define the critical Level 3 behaviors. What exactly must people do differently?
- Specify the required drivers. These are processes and systems that reinforce, monitor, encourage and reward performance of critical behaviors on the job (same workbook).
- Design the learning (Level 2) and the experience (Level 1). Only now do you decide what the program looks like.
Two of those terms are new and worth dwelling on.
Required drivers. A driver is anything outside the course that makes the new behavior more likely: coaching, job aids, manager praise, refreshers, recognition or pay for performance. Kirkpatrick Partners' own guidance gives praising an employee who is seen applying a newly learned behavior as an example, and lists coaching, mentoring and pay for performance as other drivers, noting that managers need guidance on how to use them (Kirkpatrick Partners, Rao). This is the model's direct answer to the Level 3 problem. Instead of hoping behavior transfers, you plan the support.
Leading indicators. Level 4 outcomes arrive late. Leading indicators give an early read. In the same Kirkpatrick Partners article, after a communication program, managers could survey customers about the emails they receive and watch whether escalations tied to communication problems fall.
Return on Expectations
The New World model also leans on Return on Expectations, or ROE. Kirkpatrick Partners describes it as what a successful training initiative delivers to key business stakeholders, showing how far their expectations were satisfied, and says it should be discussed with the business before a program is built (Kirkpatrick Partners, Rao).
In practice ROE is a conversation, not a calculation. You sit with the stakeholder who is paying for the program and ask what success would look like to them, in terms they'd recognize a year from now. The answer becomes the Level 4 target. It also changes who the report is for: not the L&D team, but the person whose expectations were set at the start.
Phillips and Level 5
Jack Phillips extended the structure with a fifth level, Return on Investment. The ROI Institute describes the five levels as Reaction and Planned Action, Learning, Application and Implementation, Business Impact, and ROI, where the last level compares monetary benefits to costs. It also treats isolating the effect of the program, with methods such as control groups and trend line analysis, as an essential step (ROI Institute).
So Phillips does two things Kirkpatrick's levels don't. He turns Level 4 impact into money, and he insists on isolating the program's contribution before doing so. If you need the formula, the cost categories and a worked example, they're in proving training ROI to buyers. Many organizations use Kirkpatrick for the overall frame and add the Phillips step only for a few high-cost or high-visibility programs.
Criticisms of the model
The model's popularity has always sat alongside academic skepticism. The three most-cited critiques make different points.
Alliger and Janak (1989). Thirty years after the original articles, they identified three problematic assumptions: that the levels are arranged in ascending order of information provided, that they're causally linked, and that they're positively intercorrelated. They examined how often each level appears in published evaluation studies and what the literature's correlations say about the second and third assumptions (Alliger and Janak, 1989). The practical lesson: a good Level 1 score doesn't predict a good Level 2 score, and you can't assume a chain of cause from one level to the next.
Holton (1996). He argued that the four-level system is really a taxonomy of outcomes and flawed as an evaluation model. A real model, in his view, has to specify outcomes correctly, account for intervening variables and indicate causal relationships. He proposed an alternative that accounts for variables such as motivation to learn, trainability, job attitudes, personal characteristics and transfer of training conditions, and gave participant reactions a new role (Holton, 1996). The criticism is aimed at the original model. Notice how required drivers in the New World version answer part of it, by naming the workplace conditions as design elements.
Bates (2004). Reid Bates added a later critical analysis in Evaluation and Program Planning, titled "A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence" (Bates, 2004). It is worth reading in full if you plan to rely on the model in a high-stakes setting.
Taken together, the critiques suggest a fair reading. The levels are a useful checklist of kinds of evidence. They are not a causal theory of how training produces results, and they shouldn't be used as one.
How to apply the model
A few practices follow from the sources above.
Start with the stakeholder conversation. Before designing anything, agree the Level 4 outcome with the person who'll judge the program. If you're also doing a training needs analysis, this conversation is the natural bridge between the gap you found and the evidence you'll collect.
Pick a small number of leading indicators. Two or three readable within weeks beat a dozen nobody collects.
Write the critical behaviors in observable terms. "Communicates better" can't be observed. "Confirms the customer's issue in the first reply before proposing a fix" can.
Plan the drivers, and name their owners. A driver with no owner is a hope. Decide who coaches, who checks, and what the participant's manager is asked to do.
Measure Level 2 against a baseline. Without a pre-test or equivalent, a good post-test result is hard to credit to the program, as Nickols notes.
Match effort to stakes. Full Level 3 and 4 evaluation for every course is unrealistic. Kirkpatrick Partners' own workbook frames the work as evaluating a mission-critical program, which implies choosing. Reserve the heavy treatment for programs that are expensive, strategic or politically visible. If the program is built on a structured design process such as ADDIE, evaluation planning fits naturally into its final phase.
Frequently Asked Questions about the Kirkpatrick Model
What are the four levels of the Kirkpatrick Model?
Reaction, Learning, Behavior and Results. They ask, in order, whether participants found the program favorable and relevant, whether they acquired the intended knowledge, skills, attitude, confidence and commitment, whether they perform the critical behaviors on the job, and whether the targeted organizational outcomes occurred.
Who created the Kirkpatrick Model and when?
Donald Kirkpatrick set out the four-level approach in four articles in the journal of the American Society of Training Directors, beginning in November 1959 and ending in February 1960. Later editions and updates were developed with his son James and Wendy Kirkpatrick.
What is the New World Kirkpatrick Model?
It's the updated version taught by Kirkpatrick Partners. It tells evaluators to start at Level 4 and work backward, define leading indicators and critical behaviors, plan the required drivers that support those behaviors, and only then design the learning. It also puts Return on Expectations at the center of the stakeholder conversation.
What are required drivers and leading indicators?
Required drivers are the processes and systems that reinforce, monitor, encourage and reward performance of critical behaviors on the job, such as coaching, job aids and recognition. Leading indicators are short-term observations and measurements that suggest those behaviors are on track to produce the desired results.
Does the Kirkpatrick Model measure ROI?
Not in its four levels. ROI as a fifth level was added by Jack Phillips, whose methodology converts business impact to money and compares it with fully loaded costs after isolating the program's effect. The New World model offers Return on Expectations, which is a different, stakeholder-defined idea.
What are the main criticisms of the Kirkpatrick Model?
Alliger and Janak questioned the assumptions that the levels ascend in information value, are causally linked and are positively intercorrelated. Holton argued that it is a taxonomy of outcomes rather than a true evaluation model because it doesn't specify intervening variables or causal relationships.
Do you have to evaluate every level for every program?
No. Level 1 and 2 data are cheap and usually worth collecting. Level 3 and 4 evaluation takes real effort, so it makes sense to reserve it for programs that are costly, strategic or highly visible.
Related reading
Co-Founder & COO