The danger of the BGCA example is that the training is designed to meet the things which are quantifiable, e.g., membership growth. But are those really the most important aspects of leadership? The quality of leadership requires qualitative assessment.

The mindset that the only things that matter are the things that can be quantified needs to be challenged—it’s time to move beyond the ‘intellectual poverty of quantitativism’.

ROI on training isn’t that difficult to implement. The methodology developed by Jack and Patti Phillips from the ROI Institute is both simple and effective. For those people who have not had the benefit of employing this methodology I strongly recommend it.

My concern with this article is that you can never really evaluate a training program’s impact on performance without a control group, and in this case what was the impact of each local team’s projects given that each team could choose their own project? While Training could say it had an impact on performance because all these leaders took the same class, determining the program’s impact is subjective because the projects pursued by each team were different.

In this example it appears that allowing local leadership to pursue its own direction to improve performance had as much impact as the training program itself, if not more. I taught you to fish, and you decided when where and how to fish. It would appear that this success of this program was getting local leadership to focus on the four aspects. But, isn’t this what alignment with organizational goals is about anyway?

You see, not all factors of performance success or failure are under Training’s control and therefore Training cannot say it was only the training that improved performance.

Yes, there are those who will say you can isolate the effects of training, convert that data to a monetary value and calculate a ROI—hogwash. Unless you are using control groups and control the entire process, how can you isolate the effects of training? There are too many variables in the work environment, both known and unknown, that impact performance results.

What impact does recruiting, performance support resources, informal resources, supervisors/managers, employee behavior, work priority/urgency, and work environment have on performance? All of these things have an impact because they are interconnected with the training event and Training. In other words, performance impact, positive or negative, is not just Training’s doing and to look at it as such fails to address the many influences on performance.

One approach we have used successfully at IBM is measuring “time-to-money” from our skill-building programs. This is the total elapsed time between the start of the program and the time the individual first employs the new skillset. This is based on “Level 4” of the Kirkpatrick scale of training effectiveness.

As related in this article, the value test in training for any organisation can be assessed by being given the opportunity to practice what a person has learnt during training. It is only through the application of transferred information that a company and an individual can benefit through realised knowledge.

Formally and rigourously measuring the ROI on corporate learning and development programs is not done more often because it is typically difficult and expensive to do. Almost every senior executive would like quantitative business-perfomance-based metrics. However, few are willing to pay the high price in-terms of time, energy, and sponsorship to ensure such metrics are developed and maintained.

Maybe this is not new, but from our experience as a training institute and judging by clients encouraging post-training responses, firstly, there is a need to sustain and manage the training at the end of any training programme. To ‘keep the green signal on,’; we do an on-the-job-training exercise following each of our projects.

Additionally, our clients also insisted on a positive change in their employees’ attitude towards customers, a boost in their product knowledge, and ultimately a tangible improvement in the sales figures. So to measure the real impact of training, we do a pre- as well as a post-course evaluation (mapping) of the above areas to accurately measure (determine) the impact (return) of our clients’ investment in training ventures.

One last thing to mention, that the pre-course metrics are supplied by the clients supported by CRM-generated actual KPI’s or something equally accurate.

In large corporations, these kinds of training methods are used for qualitative improvement. It would have been more useful had there been an article on training process, too.

It’s a question of ethics. If you are going to take individuals out of their normal work activities (in a classroom, virtually, coaching etcetera) you are honour-bound to have done your homework and have identified not only what the benefits will be for the person and the organisation, but also how you will be able to measure them as part of on-going continuous improvement in the design and delivery of Development solutions.

One fundamental gap in this article is the value that the learners (the Boys and Girls themselves) see in the training. That is: what’s in it for me? The value that is captured is far greater than the learning outcomes that are immediately measured (which is more of a temporal than a causal relationship).

In many cases, corporations tend to “mandate” the programs versus spending the time to understand what the learner sees as value.

I first want to say “Performance Problems Require Leadership Solutions!” Not leadership training, but rather leaders doing what they are supposed to do: lead!

That being said, I agree that “smile sheets” have the biggest impact on a training project. And then the ball drops. From that point on, the trainer goes to work to make the training more “enjoyable” and the trainees may or may not get what they “need” in the process. But as long as everyone is happy, the training goes on.

I think that evaluation doesn’t go beyond the first level because of FEAR. Plain old fear.

Fear of creating a training program that isn’t liked by all participants.
Fear of losing future participation. Create a bad class once and just try to get a budget in the future.
Fear of ridicule.
Fear of rejection.
Fear of incompetence.

All these fear drivers go from the trainer to the C-Suite. All are driven by the fear from above that they did not use training to it’s best advantage. If they did an actual study on training’s ROI, they might discover where they are making huge mistakes in their training and learning strategy. But that lesson is too risky.

Only a learning organization that can embrace all results as feedback and lessons learned for ongoing improvement will make the effort to measure the ROI for their training programs and all their programs.

Like I said “Performance Problems Require Leadership Solutions!”

As an educational technologist who has been beating the drum of measurable results of training for years now, I applaud any research being done from a management perspective.

However, I do not believe that simple ROI can be linked or measured from learning outcomes. My argument is that training is hardly ever an equity investment but a consumable expense, therefore illogical to think of it in terms of ROI.

The best way to measure training impact is to think of it as an adjunct to return on equity initiatives. It is fairly easy to determine where expenditures should be directed in order to enhance ROE, and human factors analysis (performance and training impact) should be factored into the cost equation, along with other systemic change initiatives.

ROI calculations diminish significantly or disappear when looking at bottom line measurements like ROE. But human factors (e.g. performance and training impact) as a component of ROE improvement expenditures are considerably more accurate, and much more in alignment with stakeholder goals.

For me L&D is a mindset in that you either see it as an investment that needs constant attention, i.e. ongoing coaching outside of the ‘classroom’, or, a cost that you aim to minimise at every turn. In my experience from my corporate life, very little was done to measure the real effect of L&D. A lot of money was spent and wasted because of little or no reinforcement. The key is accountability with no exceptions. Clear L&D goals linking the individual/management/dept/division with the business and/or strategic objectives and goals. Companies should know what they want to measure and how to measure it objectively. It isn’t rocket science and I am surprised more CEOs do not bang the drum and demand evidence of a return on that L&D investment instead of accepting far too many lame excuses.

This is a fairly simplistic rehash of the 4th level Donald Kirkpatrick’s 1959 model of learning evaluation. I would suppose it’s a nice primer for someone wholly unfamiliar with the field of learning evaluation, but there is a considerable body of knowledge that is more sophisticated than what is presented here.

One of the more outdated aspects of this article is the link between “training” and performance, not “learning” and performance. We know that the best approach towards learning is a blended approach, with classroom training taking on only one potential vehicle. By focusing only on the impact of training, the organization misses out on all of the other opportunities that the individual has to learn and improve.

I think the scariest thing presented here is that the organization’s view that development of its employees should be considered to be undesirable “overhead”. This is not an organization with a long, healthy future.

With 35 years of experience as an entrepreneur including 9 years in executive education I can testify that measuring impact should be an absolute prerequisite for any training program.

I am so convinced that with the programs we are running to boost intrapreneurial activity in organizations, I am prepared to link the cost of the training to the measurable results.

So far our innovative programs have generated measurable profits that are a mulitple of the cost of training.

This change of paradigm in executive education means that training can be a profit center rather than a cost center. And, when training generates business profits, line managers have a much higher appreciation for HR and people-development executives.

Good article. But go deeper and find the real cause of wastage on training. Management is ready to spend millions, but not able to give its time. Training departments are always filled with low-quality personnel. In the back of their minds, training is considered a cost, and personnel are seen from the prism of “support” services. In the rat race of the “bottom-line,” organizations forget the long-term benefits.

Solutions: Top management will have to introspect, and give real value to the training department, else such “eternal discussion” will continue.

A well-meaning learning organization had to be created. In their hearts, every employee of the training department knows how much value he or she and their department gets from top management and employees. It is a mindset issue. Have a proper mindset, and the problem will be solved.

As education gives knowledge, training gives skills. Training needs to strengthen the participant’s ability to think and act beyond his set curves. First, assess the trainee’s training needs beyond job-focused. Also explore areas like leadership, relationships, opportunities, career movements, and changing perceptiions of customer and character.

Testing real-world outcomes is crucial. Training and value-added effectiveness means that such programs and practices have the intended effect on the people or organizations toward which they are directed. In the case of training and development, key quantitative business-performance metrics should offer information on whether employees build needed skills, not just on participation in training and/or employee and management satisfaction with the training provided. Measuring only business performance metrics programs offers no insights into the training programs’ effectiveness. While surveys can be a useful tool for gauging alignment between services and the opinions of an organization’s customers, they fall short of providing the needed insights into the “real impact” ie McK “testing real-world outcomes” importancies for training programs and practices. BGCA should adjust the training curriculum to reinforce the success factors. As stated in the McK BGCA article, “picking the right metrics is the key to creating real value from training.”

I believe training and development play a crucial factor in the growth of an organization. “Leadership is a phenomenon which has to be practiced at every level” and T&D helps in achieving this goal. If we look in an Indian context for Public Sector (PSU) banks, we would find that banks primarily prescribe 2-year training programs for its aspirants before assigning any definite role. This not only enhances the knowledge and skills of aspirants but at the same time develops future leaders and laudable asset for the organization.

On the contrary, if we look at KPOs in India, most of them do not have a proper structure for training their employees except a few top brands, As they mainly focus on coming down to work and start giving results because they associate huge costs with training employees—which is a wrong notion, I believe.

As the author has also precisely mentioned that if we develop a T&D program on specific parameters, then that would not only help in saving the resources i.e., Time, money, etc. but also motivate the employees to get settled with organization and thereby reducing the attrition rate in the long run.

The comments of Professor Sloman are well articulated and not incorrect, but miss critical points. Yes, ongoing informal training and alignment are worthwhile. Let’s make sure we encourage that in our organizations. But, if one can implement a process that shows a 30 million dollar increase and a 8x ROI, it has tangible and measureable value and is worthy of our praise—even envy. Even if we agree there was considerable effort expended, there are short-term and long-term gains that would make this effort worthwhile. It might well assure this organization’s ongoing viability, not to mention improve its methods and attract others to be more involved in its mission.

As one who has done my share of interviews, I don’t think they hold a candle to hard, quantitative research and proven outcomes. I suggest we praise and encourage the BGCA’s of the world for tackling a multifaceted, complex question, clearly articulating a goal and achieving a nice return on their investment. BGCA was not “looking into a rearview mirror,” they were looking for a way to build a better car.

This is a great article. Proper, controlled analysis of senior leadership development ROI is very few and far between. The BGCA could teach a thing or two to a lot of much larger organisations.

The fundamental issue of financing leadership training is relevant to all organisations today—not just non-profits. And this shows a) measuring ROI can be done relatively costlessly and b) that leadership development programs do offer significant ROI.

A similar controlled study was done by Chicago GSB, now Booth, some 5 years ago. The report can be read at http://www.iedp.com/Teaching_Executives_to_See_Social_Capital

I absolutely agreed with the assertions made by the authors. Most companies have a training and learning and development budget but do not have a budget line item called: Measuring Return on Training Investment (ROTI)!!

The learning organization of a company needs to take the lead if the most senior leaders in the company do not see that need! It is easy to say the senior leaders don’t see it so there is no point to raise the red flag. If this is the case, then the learning organization is really letting itself down.

It will be challenging to think of the most appropriate measuring metics to link the investment in training to revenue, profit, etc. It used to be that way, too, with measuring customer satisfaction or some of the softer side of businesses.

It can be done. Just need to be more innovative and creative.

Particularly for the regular business, training is an essential activity to nurture employee skills and help in achieving better KPIs. Ideally the curriculum for such a program must be tailored for achieving set goals. Once the person is through this training, there should be a good opportunity to build skills based on knowledge gained in classroom. The assignment and subsequent assessment of results achieved is the real manner of measuring success of a training program.

Reward and recognition of people who achieve better results after such an assessment will really help to motivate larger population as well.

I like the approach taken by the authors. The key to the whole issue is “Because the program was designed to improve specific organizational-performance outcomes, the process of assessing its impact was straightforward”.

This reinforces the approach that we in the Theory of Constraints Community take, that the conflict in most actions taken (training included) is the underlying conflict between the local and the global optima. We belive that the local optmisation should be subordinated to the global, and thereby the results for the organisation will be the best. By “training for the local improvements” experts who are not in line with global requirements are created when they are not needed. They are high inventories of knowledge and skills which may or may not be needed. By focusing on the global goals, the skills and capabilities built by training are the “here and now needs” that will impact global goals needs right away.

Another very inmportant principle in the Theory of Constraints is that people behave according to the way they are measured. So, if we select global parameters to measure their perfromance they will work on global parameters.

The same applies to building organisation values and culture, which is really the reason why some companies are great companies and some companies never make it. A good article.

This is a dated and disappointing article. It is firmly in the last century tradition of spuriously ‘proving’ the value of training, justifying the intervention and hence defending the role of the training department. For a 21st century review of evaluation practice, see the report ‘The Value of Learning’ produced by the University of Portsmouth for the UK Chartered Institute of Personnel and Development.

One of the Chief Executives cited in the CIPD Report described the sort of approach used in the BCGA case as ‘driving using a rear-view mirror’. The study undertaken by BCGA must have involved an enormous amount of effort and for what? All that resulted was some marginal process improvement that could have been achieved through two day’s telephone interviews.

Modern learning and development practice involves much more emphasis on pre-activity alignment with senior stakeholder expectations—and a recognition of the importance of continuous informal learning. It is not about over-engineered number crunching after the event.

Thank you for sharing this article. It reinforces the key messages that we have struggled to convey to our clients for some time. First, the business of leadership training and development is chiefly a “C-suite” issue, not the purview of the HR Director. Second, it is key for the training to have “hooks” into the organisation’s processes, especially tying performance management into the observed practice of skills learned in the training (systems alignment). Lastly, we encourage an organisation-based measurement of the impact of training, as well as individual-based measurements such as 360 surveys. Otherwise, training is simply an overhead cost that leads nowhere.

Here in southern Africa, we find these messages to be very difficult to deliver effectively because there is a limited culture of business mentorship and coaching; senior executives often see their jobs principally as focused on results, not on organisational development. Even where they do, there are precious few models in their own professional development experience on which to base such an orientation.

Indian companies invest heavily in re-training their human capital. This is helping Indian companies to align to market requirements, and helping them to add more value to stakeholders. Training inlcudes both classroom and on-the-job training, for all human capitals in the organization.

I support nearly every point in this article. Performance metrics should be tied to training outcomes. However, many performance issues can be addressed effectively without requiring formal training. Clear expectations, clearly communicated plus job aids, coaching, and timely feedback will often trump the need to take performers off the job for training. Further, training must be designed to close a performance gap among those who really need it—and not just to inform or attempt to inspire the masses. Jim Rohn once taught: “Don’t send ducks to eagle school”. He followed that with, “Don’t send eagles to duck school—they’ll resent it.”

We saw a similar result in a large global organisation. Training was classified as a cost to be reduced or eliminated, with little thought given to the value generated by training.

We developed a training program with measurable objectives explicitly linked to the corporate objectives and identified the critical competencies required. Training investment was aligned to the development and enhancement of those competencies.

The Kirkpatrick model was adapted to provide an objective, quantified analysis of the outcomes delivered.

At the end of the first year we were able to clearly demonstrate the value delivered by the training program including a significant return on investment, quality improvements and increased employee engagement. This enabled us to obtain funding for the training program in future years.

The case of BGCA is very interesting as well as the findings from your 2010 Global Survey. Although the management development and training field has the know-how and the tools to make training effective, measuring BOTH individual participants AND at the organisational level, a higher level of evaluation in ascertaining a return on investment is rarely done. It is rather curious that survey after survey indicating similar findings, i.e., most organisations or companies do not go beyond Kirkpatrick level 1 participants’ satisfaction assessment.

A recent survey that we conducted (see http://www.adequate.org/Page%20Files/file/20091114-ISO+Focus+09+Training+Survey.pdf) found similar results amongst European companies.

I would be very happy to know what is the hindrance for not doing what we as professionals know as good practice. Equally important is why management does not require such serious operational reporting as more than 50% of the AdeQuaTE survey respondents indicated either sometimes or hardly reported on performance improvements of the trainees. Only around 10% reported that the company often looked at the return on investment of training.

Perhaps McKinsey would be interested to conduct a follow up survey on this point and find out why most organisations/companies do not assess the actual organisational capacities gained and expressed in performance matrices. Time and again, your surveys point to the top priority given by the executives or CEOs on developing organisation’s capabilities. Why aren’t they checking?

It’s nice to see someone addressing the links between training, individual performance, and organizational performance. Identifying the top performers is a tried and true method for getting at important differences in individual performance and there is where training can indeed make a valuable contribution.

The straightforward concept underlying this article is spot on! It’s an idea I often suggest in my first client meeting. But, when I propose this sort of investigation (even a less complex study), the client typically tells me, “I can’t afford it.” Or, “I know what we need; I work here.” As you point out, the ROI is staggering when the meaningful metrics are determined. I’ve even proposed that my fee is dependent on cost saving or increased profit, but I’ve had few takers. Conclusion: organizations talk the game of long term strategies and metrics-focus, but are acting short term. Sad.

The article is rather generic in its approach. Perhaps it would have been more helpful if some insight was offered on the deliverable aspect of training. For example, skills are developed by practice not by reading about them. How do the trainers include this dimension into an interactive session and then measure the impact of the practice on a “before” and “after” mode? The improvement achieved can then be converted into a dollar value. And so on.