In November 1959, Donald Kirkpatrick published a series of seminal articles on training evaluation in the ‘Journal of the ASTD’. In the fifty years since, his thoughts (Reaction, Learning, Behavior, and Results) have gone on to evolve into the legendary Kirkpatrick’s Four Level Evaluation Model and become the basis on which learning & development departments can show the value of training to the business. How has the model evolved over fifty years, is it still relevant? As designers of learning, have we applied the model with Don’s intent?
Jim Kirkpatrick (is SMR USA’s vice president of global training and consulting, and presents workshops for and provides consulting to Fortune 500 companies around the world) and Wendy Kirkpatrick (is a director of Kirkpatrick Partners, LLC and a certified instructional designer) have written a paper which honors Don, and takes a fresh look at Kirkpatrick’s Four Levels.
The whitepaper delves into the many issues associated with the use and implementation of the model to evaluate training. The authors espouse Five Foundational Principles
- The end is the beginning – An incisive comment: ‘For decades, practitioners have attempted to apply the four levels after a program has been developed and delivered. It is difficult, if not impossible, to create significant training value that way.’ They go on to add it’s important to distinguish a plan to build effective training & the evaluation methodology from the actual collection of data.
- Return on Expectations (ROE) is the ultimate indicator of value – Despite what they might say, training professional tend to jump into the task of designing and developing learning without a proper needs assessment. Learning professionals do nothing to track the expectations of the business stake holders. Without this critical assessment, it’s not possible to map expectations to observable and measurable success outcomes.
- Business partnership is necessary to bring about positive ROE – The authors point out that they do not believe that training events in and of themselves can deliver positive bottom-line outcomes. They go on to add that research suggests that 70% of learning happens on the job in many different ways. In this context, they emphasize the role of the supervisor or manager after the training event. Reinforcement and coaching by such individuals directly correlates to improved performance and positive outcomes.
- Value must be created before it can be demonstrated – Research has identified statistical correlations between the four levels. In short, it suggests that providing excellent training does not lead to significant transfer of learning to behavior and subsequent results without deliberate and consistent reinforcement. They mention that learning designers are putting most our time on designing, developing and delivering training and getting only about a quarter of the result. This is mainly because we spend virtually no time on follow-up reinforcement activities to promote behavioral change that leads to results.
- A compelling chain of evidence demonstrates your bottom-line value – Authors mention L1 and L2 as consumptive metrics, L3 and L4 are classed as impact metrics. While working at L1 and L2, we are talking only about costs – because we only talk about number of programs, attendees (typical L1 and L2 data). If we had the data for L3 and L4 metrics, the data would point to the value the training delivers. They go on to mention that a chain of evidence connects the four levels and will actually show the contribution the training has made to the business.
Other than listing and describing the Five Foundational Principles, the authors also introduce a revised new-look Kirkpatrick Model. The authors mention that the model retains the fundamental ideas that Don (& the authors) have been communicating, it’s now said to offer a more complete illustration so both learning professionals and their business partners can create maximum training value. They further give detailed information about the model and various examples. The last example they give of ‘The Window Washers’ makes an excellent case for ‘not just training’.
Anyone interested in evaluating learning programs based on Kirkpatrick’s Four Level model should read this paper; if offers great insights, I’d recommend it.
