In this article, Sydney Savion, Chief of Education Strategy, Dell EMC Education Services, explores ROI and operational efficiency.
This is an interactive article, and we want your feedback! We’ve embedded a quick question at the end, to allow you to give your feedback and we’ll share in a future piece.
It is widely known that Chief Executive Officers (CEO) prize business impact, business alignment and return on investment (ROI) given companies are measured by their performance in the marketplace and those that set themselves apart count on learning and development (L&D) to build and measure the workforce capabilities and skills necessary to deliver business results.
The internet is replete with information on learning measurement, evaluation, and analytics. Overwhelmingly most L&D organizations measure and report ROI in terms of operational efficiency i.e. number registered, average number of staff per learner, hours per learner, and cost per student hour. But wait! If CEOs are relying on L&D to build and measure the workforce capabilities and skills necessary to deliver business results, what measurements should L&D report on to demonstrate value in terms that CEOs can appreciate?
“GUNG HO Effect”
The scene opens at an auto plant in a small town in Pennsylvania, which provided most of the town’s jobs. For nearly a year it’s been closed. Hunt Stevenson, a former foreman travels to Tokyo, Japan to try to persuade the Assan Motors Corporation to reopen the plant. The Japanese company agrees. Upon their arrival in the U.S. they implement many changes. The workers are held to more regimented standards of efficiency, effectiveness, and quality. However, their poor work ethic and lax attitude toward quality control persists, resulting in marginal output. Japanese management is perturbed by the results and sees no reason to keep the plant open. In an attempt to solve the problem, Stevenson brokers a deal with Japanese executives to produce 15,000 cars in one month, hence reaching productivity levels of the best Japanese auto plant. If the workers succeed they get raises and more jobs will be created for the remaining unemployed workers in the town, if they fail, they get nothing and the plant closes. Motivated, the workers work diligently toward their goal and pursue it with the drive that Japanese managers had encouraged. They soon realize time was running out and they would only be able to produce 13,000 cars by the time of the final inspection. So Stevenson and the workers lined up piecemeal cars hoping to dupe the Japanese executives. The ploy is magnified when the car Stevenson attempts to drive away in falls apart. Gung Ho is a1986 American comedy film. But, many L&D organizations are faced with this challenge–this kind of “Gung Ho effect”. They create an overabundance of offerings, present training ROI to CEOs using operational efficiency data but come up short on showing business impact. Not surprising then that Bersin by Deloitte found when it comes to L&D, nearly 92% of companies say the number one challenge they face is measurement.
What do CEOs Want?
In a study comparing what L&D organizations currently measure compared to what CEO want, Jack and Patti Phillips of the ROI Institute reported some resounding findings:
More than 50% of L&D organizations measure and report efficiency measures such as the number of participants, the number of courses, satisfaction, utilization rate and cost. On one hand, L&D is giving the CEOs what they want, considering over 80% of the CEOs want some of this information. On the other hand, more than 60% want to know about the application on the job and nearly 100% want to see L&D data coupled to business impact. Currently, only 8% get it.
L&D continues to be plagued by the challenge of formulating a plan of execution to measure the impact of what’s fast becoming petabytes of content stored and pushed to employees. Jack Phillips of the ROI Institute asserts, “CEOs want to see value in terms that they can appreciate. They view the value of learning and development in terms of business impact, business alignment, and return-on-investment. They do not see value in the inputs (how many people attended training) or reaction (participants rated overall satisfaction 4.2). There is very little data at the business contribution level presented to them, and yet, that is their most important data set.”
Moreover, executives believe developing workforce capabilities and skills are essential to driving business outcomes. CEB found nearly 90% think so. To compound this issue employees often take excursions to non-L&D content and sources to learn what then need, when and where they need it. So no surprise that unnecessary and/or poor-quality learning content accounts for nearly 2/3 of L&D cost, according to CEB. It’s no secret why learning measuring impact is top of mind for CEOs. This is a clear and compelling call to action for L&D to start taking cues from the business and by extension the CEO. Not to worry, there seems to be a slow awakening and global shift in approach taking root to moderate the “Gung Ho effect”.
Observable Shift In Approach
If you want to think about how to measure training, you need to think about it in terms of how well it supports business initiatives that have financial goals. -Josh Bersin
Among others, Jack & Patti Phillips of the ROI Institute have observed that the L&D community is responding with changes in its approach to measurement, evaluation, metrics, and analytics. The community is starting to think in terms of balancing growth and risk, and asking what measurements, capabilities, and talent are required to impact the business. Much like Abraham Maslow’s Hierarchy of needs based on a theory of human motivation, a CEO’s desire to know about application, impact, and ROI of L&D is based on fundamental psychological needs. In order to show concrete value, the L&D organizational ecosystem must be hyper-focused on application, impact, and ROI. Here are five measurement trends pointing to a gradual shift towards this approach:
- Measurement strategy: Building a house starts with a blueprint. Likewise, to show impact, measurement must start with a good plan. Develop a comprehensive plan that comprises the business goals, key performance indicators, measurable data sets, segments, targets of success, calculations, how and when things will be measured.
- Cues from the Business: The number one reason L&D exists is to help the business deliver results. Take cues from the business and by extension the CEO and deliver on the business strategy key drivers. Don’t stay stuck in the “Gung Ho effect”, lean in and give the CEO what he/she wants to know about: application, impact, and ROI.
- Timely and Relevant Delivery: L&D’s forte is designing, developing, delivering, and curating content. In fact, there tends to be a saturate of offerings, often short on impact. Based on business goals and key performance indicators, identify learning solutions and metrics that would indicate success. Moreover, ensure it is relevant to the job and equips people with skills that are required, when and where they required to move the business ahead.
- Learning Content Lifecycle: Best practice in learning content lifecycle management is to use a performance consulting approach. This means at the front end of the learning content lifecycle process partner with the business to conduct a needs assessment, identify the root cause of the business problem, and establish measures of success. Additionally, set the framework for data requirements by formulating reaction, learning, application, impact and ROI objectives.
- Measurement Culture: Organizational culture is a powerful, hidden force of a company and by extension an L&D function. Take stock of these forces because they influence group behavior. Build a measurement culture. Establish standard operating procedures, governance, and rewards for the execution of the measurement strategy. Moreover, foster a culture that gets everyone thinking about application, business impact, accountability, actionable data, measures, analytics, business insights, metrics, and ROI.
The so-called “Gung Ho effect” of creating an overabundance of offerings and reporting ROI with only operational efficiency data, fall short on measures that show how L&D is building workforce capabilities and skills that deliver business results. There are clear and convincing findings that point the L&D community in the direction it ought to go. Besides measuring operational efficiency,
L&D should be reporting on application on the job, business impact data, and measurements that demonstrate value in terms that CEOs can appreciate.
How do you think L&D should measure ROI?
Latest posts by Dr. Sydney Savion (see all)
- “GUNG HO Effect” of Only Measuring Operational Efficiency - March 16, 2017
- Knowledge Sharing: Case For and Against L&D Ownership - February 7, 2017