+1 617.391.0425 Mind Tools Home

You are here:

Request a Demo Contact Us

L&D’s Potential Fairytale Ending

Bob Little 

November 6, 2015

iStockphoto/alga38Once upon a time, those in the L&D function had confidence in the certainties underpinning their professional lives – any training program helped build “better,” more productive people, and learning any new thing helped you develop your knowledge base, skill set, and as a person.

This was a happy time because L&D professionals knew they were always making a positive contribution to their organization and its people, even if others didn’t recognize it.

Then came the realization that learning programs should be evaluated to ensure they were producing the results they claimed. From that developed the belief that an organization should calculate its return on expectations (ROE) to indicate any L&D program’s value. Another school of thought then suggested that, in this finance-governed world, we should calculate the return on investment (ROI) rather than the ROE – and that prompted another debate about the most appropriate and acceptable way to calculate ROI.

Life in L&D wasn’t simple any more.

L&D professionals now consider such questions as, “How should you measure the strategic value of learning? Is ROE a more effective measure than ROI? How do you take proper account of all the variables that affect workers’ performance? How do key stakeholders expect learning to add value? How can you link investment in learning to strategic priorities?”

It was November 1959 when, via the then ASTD journal, Donald Kirkpatrick published his views on training evaluation. He emphasized the value of evaluation if the purpose of training is to improve results by changing behavior. Forty eight years later, Donald and his son, Jim, wrote a book, “Evaluating Training Programs – The Four Levels,” advocating presenting the value of training to “maximize the meaningfulness to the hearts and minds of your internal stakeholders.” They added that this should help L&D professionals get larger training budgets and commitment to carry out the changes needed to achieve organizational objectives.

Then, Jim and his wife, Wendy, further refined the concept of training evaluation, developing and defining ROE as a holistic measurement of the qualitative and quantitative benefits realized from a program brought about through a package of interventions, with formal training typically being the cornerstone.

To determine ROE, they say, L&D professionals must clarify and refine key business stakeholders’ expectations, while trying to convert those expectations into observable, measurable business or mission outcomes. L&D professionals should also encourage supervisors and managers to prepare learners for training and, after training, reinforce newly learned knowledge and skills with support.

While agreeing that learning’s outcomes are affected by many factors, Jim and Wendy Kirkpatrick argue that, if you don’t know and measure which factors help the transfer of learning to behavior, you can’t take credit for the program’s success.

The concept of ROI in learning – comparing a program’s monetary benefit to the organization with the money invested in it – has its origins in the work of Jack Phillips in the 1980s. Phillips wanted the L&D community to move beyond Kirkpatrick’s Level 4 evaluation to a “Level 5,” where a program’s success was measured in financial terms. A key part of Phillips’ ROI methodology is isolating the effects of the program, since, without this, the L&D function can’t take credit for the improvements identified in the Kirkpatrick Level 4 evaluation.

So there are some differences between ROE and ROI:

  • ROE focuses on Kirkpatrick’s four levels of evaluation, while ROI encompasses these and adds a fifth level.
  • ROE is a collaborative agreement intended to unite an organization in working towards a common goal. ROI is a summative measurement that tries to isolate the efforts of each business unit within the organization.
  • Finding ROI requires data not found in the other evaluation levels. It also means adjusting benefits and costs to compare them on the same scale – and then comparing this with other programs’ figures.
  • ROI supporters believe executives want to see how all factors uniquely influence outcomes.  Conversely, ROE supporters promote the business partnership approach, focusing on collective efforts. They factor in – not out – contributions from internal partners, enabling L&D professionals to be seen as strategic business partners.

Both ROE’s and ROI’s supporters believe that, before embarking on an L&D program, you should determine what “success” would look like to stakeholders – and, in any case, this should form part of the training needs analysis (TNA) undertaken at the project’s outset. ROE supporters say this enables you to focus training reinforcement and evaluation to meet those expectations, while ROI supporters argue that the “success” definition provides the benchmark against which the eventual ROI calculation is measured.

While ROE reports can include an ROI calculation if stakeholders wish, they address less “black and white” issues, such as “an overall increase in sales,” “a reduction in wastage,” or “an improvement in employee morale or loyalty.”

ROI aims to isolate training benefits and calculate the financial return on an event. However, the real questions are, “Can you reliably isolate the effects of training from other business and external factors? And are all the returns you’re looking for quantifiable?”

“This is tricky to do for behavioral training – especially if you try to prove that only the training program is responsible for people’s behavior changes over time,” says Emma Browning, managing director of Emma C Browning Ltd, an organization providing professional, commercial and practical HR solutions. “Many other factors, such as organizational environment, current business climate, self-learning, on-the-job development, and peer pressure, come into play.”

“Calculating ROE allows learning’s intangible benefits to be recognized as much as the financial ones,” says David Bligh, head of personnel at Firstco Ltd, an international consultancy and system integrator. “If you’ve agreed, at the outset, the required behavioral changes and the indicators for those behavioral changes, Kirkpatrick methodology enables you to present results at stages throughout your evaluation process. You should be able to show your stakeholders that your development initiatives are moving the organization in the right direction and that required behavior change and results are being achieved both immediately and continually as a result of the training.”

So, should you champion ROE or ROI? It depends – among other things – on your goals, the metrics you intend to use to measure success, and your stakeholders’ needs. What’s undeniable is that, in this brave, new L&D world, you need to know – and be able to use wisely – both approaches if you want to live happily ever after.

Leave a Comment

Your email address will not be published. All fields required.

View our Privacy Policy.