Delivering Training and Measuring What Actually Changes: Learning Transfer and the Kirkpatrick Model Under Scrutiny

"Training delivery" covers more ground than the usual shorthand of classroom versus e-learning. It includes instructor-led sessions, self-paced online modules, blended programs that combine the two, simulations, on-the-job coaching, and job aids consulted at the moment of need. For decades, debates about which of these formats works best treated the medium itself as the active ingredient, as though putting the same content into a classroom versus a screen would predictably change how much a trainee learns. Richard Clark's (1994) article in Educational Technology Research and Development, "Media Will Never Influence Learning," challenged that assumption directly, arguing that a delivery medium is, in his words, a vehicle that carries instruction rather than a cause of learning in its own right, in the same sense that the truck delivering groceries does not itself change anyone's diet. Clark's claim is specific: holding the instructional method constant, the format used to deliver it, in a room or on a screen, should not by itself produce a learning difference. The claim was contested at the time by researchers such as Robert Kozma, who argued some media possess capabilities, simulation, interactivity, that certain methods cannot easily replicate elsewhere, and the disagreement between the two positions, sometimes called the great media debate, was never fully settled on theoretical grounds. What later large-scale evidence shows is closer to Clark's position than to a simple "better format" story.

Traci Sitzmann, Kurt Kraiger, David Stewart and Robert Wisher's (2006) meta-analysis in Personnel Psychology, "The Comparative Effectiveness of Web-Based and Classroom Instruction," pooled 96 studies and 19,331 trainees to compare web-based instruction directly against classroom instruction. Web-based instruction came out about 6% more effective for teaching declarative knowledge, facts and concepts a trainee needs to state or recognize; the two formats were statistically equivalent for teaching procedural knowledge, skills a trainee needs to perform; and trainees reported equal satisfaction with both. The detail that matters most for Clark's argument sits inside the moderator analysis: when the same instructional methods, the same practice, feedback and content sequencing, were used in both the web-based and classroom versions of a course, the small effectiveness gap between formats disappeared almost entirely, with the instructional method delivered through either format accounting for most of what the comparison actually found.

Winfred Arthur, Winston Bennett, Pamela Edens and Suzanne Bell's (2003) meta-analysis in the Journal of Applied Psychology, "Effectiveness of Training in Organizations," reinforces the same point from a different angle. Looking across a large body of organizational training studies and all four Kirkpatrick-type evaluation criteria discussed later in this lesson, they found medium to large average effect sizes, roughly .60 for reaction, .63 for learning, .62 for behavior and .62 for results, and found that training method and the type of skill or task being trained were related to effectiveness, while delivery format alone was not the dominant factor. One further finding from that study is worth stating because it previews a theme this lesson returns to directly: only about 6% of the training programs they reviewed reported having conducted a formal needs assessment beforehand, and doing so was not consistently associated with stronger outcomes in their sample, a reminder that even well-measured training effectiveness research cannot fully substitute for the front-end diagnostic work Lesson 42 covers.

If the format itself rarely drives the result, the more useful design question is not which medium to choose but which instructional method the content actually needs, and whether the chosen medium can deliver that method well. Procedural and psychomotor skills generally need deliberate practice with feedback close in time to the attempt, a requirement a well-built simulation or a live supervised practice session can satisfy and a passive video lecture usually cannot, regardless of whether either one is labeled classroom or e-learning. Declarative content, policy knowledge, product facts, compliance rules, tends to transfer reasonably well through either format once the method includes retrieval practice and spaced repetition rather than a single read-through. This reframing does not mean every format choice is interchangeable in every situation. Complex interpersonal skills, difficult conversations, negotiation, coaching, still commonly rely on live facilitation for the real-time feedback a skilled observer provides, and the research comparing formats for that specific category of content is thinner and less conclusive than the declarative and procedural comparisons above, a genuine limitation worth stating rather than glossing over. Organizations that choose blended delivery, often for cost or scheduling reasons as much as pedagogical ones, are not automatically buying a learning advantage simply by combining formats; the advantage, where it exists, still comes from whichever instructional methods the blend actually contains.

Once a training program has been delivered, the obvious next question is whether it worked, and the dominant answer to that question for more than sixty years has been a four-level framework developed by Donald Kirkpatrick. Kirkpatrick introduced the model not as a single paper but as a four-part series, "Techniques for Evaluating Training Programs," published across two issues of the Journal of the American Society of Training Directors in late 1959 and early 1960, one article per level, before consolidating and popularizing the framework decades later in his 1994 book Evaluating Training Programs: The Four Levels. The four levels are Reaction, how trainees felt about the program and whether they found it relevant and engaging; Learning, the measurable increase in knowledge, skill or attitude the program produced, typically assessed through a test or demonstration at or near the end of training; Behavior, the degree to which trainees actually apply what they learned once back on the job; and Results, the organizational outcomes, productivity, quality, cost, turnover, that the training was ultimately meant to influence.

The model's dominance is not a mystery once its practical appeal is separated from its empirical standing. It is easy to teach, easy to remember, and maps naturally onto the data organizations already find easiest and cheapest to collect: a reaction survey takes minutes to administer, a knowledge test shortly after training is a routine extension of the session itself, while measuring behavior change requires observing people back at work weeks or months later, and measuring results requires isolating training's contribution from everything else happening in the business at the same time. George Alliger and Elwood Janak's (1989) review in Personnel Psychology, revisiting the model thirty years after its introduction, reached a conclusion that explains the model's staying power better than any claim about its scientific validation: the framework provides a useful vocabulary and a reasonable taxonomy for organizing training criteria, a genuinely valuable function even if the causal assumptions built into how most people use the model do not hold up.

Alliger and Janak's (1989) article, "Kirkpatrick's Levels of Training Criteria: Thirty Years Later," identified three assumptions implicit in how the model gets used in practice, even though Kirkpatrick's original articles never claimed to have proven them. The first is that the levels are arranged in ascending order of informational value, so that a Results measure is assumed to be inherently more meaningful evidence than a Reaction measure. The second, the one with the most direct practical consequence, is that the levels are causally linked, so that a positive reaction is expected to produce learning, learning is expected to produce behavior change, and behavior change is expected to produce results. The third is that the levels are positively intercorrelated as a matter of empirical fact, not just theoretical expectation. Alliger and Janak's review of the available research found very limited support for any of the three, and found specifically that positive reactions to training do not reliably predict how much trainees actually learn, a finding that cuts directly against the common practice of treating a high satisfaction score as a proxy for training quality.

Alliger, Scott Tannenbaum, Winston Bennett, Holly Traver and Allison Shotland's (1997) follow-up meta-analysis in Personnel Psychology, "A Meta-Analysis of the Relations Among Training Criteria," extended the 1989 review with a larger body of accumulated studies and added a useful distinction rather than simply confirming or overturning the original finding. Reaction measures, it turned out, are not all the same kind of thing. "Affective" reaction items, how much trainees enjoyed the experience, correlated weakly with learning and with later job performance, essentially confirming Alliger and Janak's original skepticism. "Utility" reaction items, whether trainees judged the content as useful and relevant to their actual work, correlated meaningfully better with both learning and transfer than the affective items did, though still well short of the strong, automatic relationship the model's causal framing implies. The honest summary is not that reaction data is worthless, a conclusion the evidence does not support either, but that which specific reaction question gets asked matters more than whether a reaction survey happens at all, and that none of this establishes the kind of reliable causal chain the four-level hierarchy is commonly assumed to represent.

Reid Bates's (2004) article in Evaluation and Program Planning, "A Critical Analysis of Evaluation Practice: The Kirkpatrick Model and the Principle of Beneficence," pressed the critique further on two fronts. First, he challenged the assumption, built into how the levels are usually ranked and taught, that a higher-numbered level automatically provides superior evidence to a lower one; a Results measure confounded by factors unrelated to training, a market shift, a staffing change, a new competitor, is not self-evidently better evidence of training's effect than a carefully designed Learning assessment, and treating level number as a stand-in for evidentiary quality can lead an organization to over-invest in results-chasing while under-investing in rigorous learning measurement. Second, and more specific to complex organizational settings, Bates argued the model tends to ignore contextual and systemic factors, organizational culture, resource constraints, competing priorities, that shape whether training can plausibly produce the outcomes being measured at all, a concern he raised with particular force in public-sector and nonprofit contexts where those constraints are often severe and largely outside a training function's control. His title's reference to beneficence reflects a specific concern that evaluation practice should serve the people the training is meant to help, not simply generate the tidiest set of numbers for a dashboard.

It is worth stating the tension plainly rather than resolving it artificially. The Kirkpatrick model is simultaneously one of the most widely used frameworks in learning and development and one of the most consistently challenged by the research literature that has examined its central mechanism. Both things are true at once, and they are true for different reasons. The model survives in practice because it gives a shared vocabulary, a defensible-sounding structure for a training report, and a way of organizing measurement effort that does not require a statistics background to use. It survives in spite of the evidence on its causal chain, not because that chain has been empirically vindicated. An organization that uses the four levels as an organizing taxonomy, collecting reaction data because it is cheap and worth having, learning data because it indicates whether the content landed, behavior data because that is where training actually earns its budget, and results data when it can genuinely be isolated, is using the model defensibly. An organization that assumes a 4.6 reaction score is evidence that behavior will change is relying on an assumption the research keeps failing to confirm.

If Kirkpatrick's levels describe what to measure, Timothy Baldwin and J. Kevin Ford's (1988) article in Personnel Psychology, "Transfer of Training: A Review and Directions for Future Research," explains why the gap between Learning and Behavior opens up in the first place. Their model organizes the determinants of transfer into three categories of training input. Trainee characteristics include ability, personality, and motivation to transfer, a trainee's own intention and expectation that applying the new skill is worthwhile. Training design covers the instructional principles built into the program itself, whether it includes enough varied practice, whether content is sequenced to match how the skill will actually be used, and whether the training environment resembles the job environment closely enough that what is learned generalizes beyond the training room. Work environment covers everything the trainee returns to afterward: whether a supervisor reinforces the new behavior, whether peers model it, and whether the trainee ever actually gets the chance to use it.

The model's most distinctive contribution is separating transfer into two genuinely different outcomes rather than treating it as one event. Generalization is whether a trained skill extends to situations somewhat different from the exact scenarios practiced during training, a supervisor trained on one type of customer complaint applying the same underlying skill to a different complaint they never specifically rehearsed. Maintenance is whether the behavior persists over time once the novelty of the training fades and old habits reassert themselves under normal workload pressure. A program can produce excellent generalization and still fail on maintenance, or the reverse, and Baldwin and Ford's framework treats that distinction as a design question, not a detail.

A number repeated constantly in training literature claims that only about 10% of training content actually transfers to on-the-job behavior change. The figure traces back to Dana Georgenson's (1982) short article in the Training and Development Journal, "The Problem of Transfer Calls for Partnership," which opened with a rhetorical question putting the estimate in the mouth of an unnamed, composite training director, "I would estimate that only 10 percent of content which is presented in the classroom is reflected in behavioral change on the job." Georgenson cited no study behind the number because none existed; it was a plausible-sounding complaint used to open an argument, not a research finding, and subsequent citations by other authors through the 1980s repeated it with growing confidence until it read as an established fact rather than an unsourced estimate.

The actual survey evidence tells a real story, just not that specific one. Alan Saks and Monica Belcourt's (2006) study in Human Resource Management, "An Investigation of Training Activities and Transfer of Training in Organizations," surveyed Canadian training and development professionals, who estimated, on average, that 62% of training was applied to the job immediately afterward, dropping to 44% at six months and 34% at one year. Kevin Ford, Stephen Yelon and Abigail Billington's (2011) review in Performance Improvement Quarterly named the Georgenson figure a delusion and pulled together better-sourced estimates, converging on non-transfer rates of roughly 38% immediately, rising to 56% at six months and 66% at one year. The two sets of figures are not identical, both rely on self-reported estimates rather than direct behavioral observation, and neither should be repeated as a precise universal rate. What they agree on is the pattern a meaningful share of training does not survive the return to the job, and the share that fails grows substantially as time passes, the empirical puzzle Baldwin and Ford's model exists to explain.

Returning to this lesson's opening case, the branch-supervisor training that scored well on Reaction and Learning but produced no change in the audited error rate, Baldwin and Ford's three categories give a structured way to ask what actually happened rather than assuming the content itself was simply bad. A trainee-characteristics explanation would ask whether newly promoted supervisors, still learning their new role's basic demands, had the motivation and bandwidth to prioritize a new approval checklist over everything else competing for their attention in the first weeks on the job. A training-design explanation would ask whether the two-day session included enough realistic practice applying the checklist to the kind of ambiguous, time-pressured cases supervisors actually face, or whether it mostly covered the checklist's logic in the abstract. A work-environment explanation, the category Baldwin and Ford's review and the research that followed it treat as the most consistently neglected, would ask whether branch managers back on the floor actually expected supervisors to use the new procedure, whether anyone checked that they did, and whether the old, faster, non-compliant shortcut was still implicitly tolerated because it kept queues moving. All three explanations are plausible from the Reaction and Learning data alone; nothing in a 4.6 satisfaction score or a passed quiz distinguishes between them.

Lisa Burke and Holly Hutchins's (2007) review in Human Resource Development Review, "Training Transfer: An Integrative Literature Review," synthesized the substantial body of research that had accumulated since Baldwin and Ford's original 1988 review and organized the work-environment determinants of transfer into five specific influences: opportunity to perform the new skill on the job, supervisor and peer support, a visible strategic link between the training and the organization's actual goals, a transfer-supportive climate more broadly, and accountability, some mechanism that checks whether the new behavior is actually being used. Brian Blume, Kevin Ford, Timothy Baldwin and Jason Huang's (2010) meta-analytic update in the Journal of Management, "Transfer of Training: A Meta-Analytic Review," quantified how strongly several of these factors predict transfer across the accumulated studies, with transfer climate and the trainee's own pretraining self-efficacy and cognitive ability emerging as some of the more consistently supported predictors. The practical implication for a training function is specific rather than general: the conditions a trainee returns to after the session, not the session's content quality alone, explain a substantial share of whether a skill survives, which means a training department that owns only the classroom and hands off responsibility the moment the session ends is managing less than half of what actually determines transfer.

Kirkpatrick's four levels end at Results, an organizational outcome attributable, at least in part, to training. They do not, on their own, answer a question most finance and executive stakeholders eventually ask: was the training worth what it cost. Jack Phillips's ROI methodology adds a fifth level that takes a measured Results figure, isolates the portion of that change plausibly attributable to the training itself rather than to other things happening in the business at the same time, converts it into a monetary value, and compares that value against the program's full cost to produce a return-on-investment figure. Doing that conversion credibly is a substantial methodological undertaking in its own right, isolating training's contribution, choosing a defensible monetary conversion for outcomes that are not naturally financial, and deciding how to treat costs that are not fully attributable to the ROI calculation, and it is deliberately not built out in this lesson. Lesson 45 takes up Phillips's methodology and the broader question of learning ROI directly; what matters here is the logical dependency, a credible ROI figure requires a credible Results figure underneath it, and a Results figure in turn depends on the kind of honest Behavior-level measurement this lesson has argued most organizations skip.

Three threads run through this lesson and converge on the same practical point. The delivery-modality research says the format a program is delivered through matters far less than the instructional method it carries, which means choosing e-learning, classroom, or a blend should be driven by what the content and the trainee population actually need, not by an assumption that one format is categorically stronger. The Kirkpatrick critique says the four levels remain a useful organizing vocabulary without being a validated causal chain, which means a strong Reaction or Learning score should be reported as exactly what it is, evidence about reaction or learning, not quietly treated as a stand-in for evidence about behavior or results. The transfer-of-training research says that whether a trained skill survives the return to work depends heavily on trainee motivation, training design, and above all the work environment the trainee returns to, which means a training function that stops owning the problem the moment the session ends is abandoning the variable the evidence says matters most.

The uncomfortable implication these three threads share is not a design flaw in Kirkpatrick's model, Baldwin and Ford's model, or any newer framework meant to replace them. It is that measuring Behavior and Results honestly, rather than reporting the Reaction and Learning numbers that are cheap, fast, and usually flattering, tends to produce findings a training function's sponsors may not want to hear, including, sometimes, that a well-liked, well-reviewed program changed almost nothing on the job. No model refinement resolves the organizational incentive to keep measuring what is easy and favorable instead of what is hard and occasionally damning; that choice is made by whoever decides what a training evaluation report actually includes, not by which four-letter or five-level model sits at the top of the page.

Select a real or realistic training program you can describe in reasonable detail, drawn from your own organization (disguised if needed), a documented case, or a program you research directly. Using that program as your basis, prepare a report of 2,000 to 2,500 words that:

1. describes the program's current or planned delivery format and evaluates, using Clark (1994) and Sitzmann, Kraiger, Stewart and Wisher (2006), whether the format choice is justified by the content and instructional methods involved, or driven mainly by cost or convenience;
2. states explicitly which Kirkpatrick levels the program currently measures or plans to measure, and identifies, using Alliger and Janak (1989) and Bates (2004), which causal assumptions a stakeholder reading that evaluation plan would implicitly be relying on;
3. applies Baldwin and Ford's (1988) three input categories, trainee characteristics, training design, and work environment, to identify the program's most significant transfer risk, with reasoning specific to the program, not generic statements that transfer is important;
4. proposes one concrete intervention addressing that risk, drawing on at least one of Burke and Hutchins's (2007) five work-environment influences; and
5. states explicitly whether a Phillips-style ROI calculation (Level 5) would be appropriate for this program at its current stage of measurement, and explains why, using the logical dependency this lesson described between Results and ROI.

Use at least five academic sources, including at least four from this article's reference list.

Assessment rubric (100 marks)

1. Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology, 42(2), 331–342. doi.org/10.1111/j.1744-6570.1989.tb00661.x
2. Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology, 50(2), 341–358. doi.org/10.1111/j.1744-6570.1997.tb00911.x
3. Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88(2), 234–245.
4. Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), 63–105. doi.org/10.1111/j.1744-6570.1988.tb00632.x
5. Bates, R. (2004). A critical analysis of evaluation practice: The Kirkpatrick model and the principle of beneficence. Evaluation and Program Planning, 27(3), 341–347.
6. Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta-analytic review. Journal of Management, 36(4), 1065–1105.
7. Burke, L. A., & Hutchins, H. M. (2007). Training transfer: An integrative literature review. Human Resource Development Review, 6(3), 263–296.
8. Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29.
9. Ford, J. K., Yelon, S. L., & Billington, A. Q. (2011). How much is transferred from training to the job? The 10% delusion as a catalyst for thinking about transfer. Performance Improvement Quarterly, 24(2), 7–24.
10. Georgenson, D. L. (1982). The problem of transfer calls for partnership. Training and Development Journal, 36(10), 75–78.
11. Kirkpatrick, D. L. (1959-1960). Techniques for evaluating training programs (Parts 1-4). Journal of the American Society of Training Directors, 13(11), 3–9; 13(12), 21–26; 14(1), 13–18; 14(2), 28–32.
12. Saks, A. M., & Belcourt, M. (2006). An investigation of training activities and transfer of training in organizations. Human Resource Management, 45(4), 629–648.
13. Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative effectiveness of web-based and classroom instruction: A meta-analysis. Personnel Psychology, 59(3), 623–664.

Verification notes: all thirteen references were checked via web search against publicly indexed bibliographic records, including Wiley Online Library abstract pages, APA PsycNet-indexed journal pages, ERIC, Semantic Scholar, and Scientific Research Publishing's reference database. The reported figures (the .60-.63 effect sizes from Arthur et al., 2003; the 6% web-based advantage from Sitzmann et al., 2006; the 62%/44%/34% transfer-decay figures from Saks & Belcourt, 2006; the 38%/56%/66% non-transfer figures from Ford, Yelon & Billington, 2011) are drawn from indexed summaries of each study rather than a full-text read of every article; no statistic or finding beyond what is reported in these sources is claimed. Georgenson's (1982) original 10% figure is deliberately reported as an unsourced rhetorical claim, not a verified research finding, per a secondary historical account (gothamCulture, 2026) tracing its citation history. The Nokshi Garments Group case, the tables, the decision scenario, the myth/fact items, and the synthesis connecting delivery evidence, the Kirkpatrick critique, and Baldwin and Ford's transfer model are original teaching material for this series.

Lesson 45 continues Module 09: once Behavior and Results have been measured honestly, as this lesson has argued they rarely are by default, the next step is converting that evidence into a defensible return-on-investment figure using Phillips's methodology, alongside career development, mentoring, and coaching as the other mechanisms this module covers for sustaining what training alone often cannot.