Some Challenging Aspects of Evaluation
Once we have pinpointed the scope of our evaluation, we have a litany of choices regarding how to perform it. There are formative (indicative of what has been learned through the progress of course development, this is part of the instructional process) and summative (given periodically post-training to determine at a particular point in time what students know and do not know) evaluations; there is long-cycle (measuring the performance of some in order to predict, plan or otherwise influence the performance of others) versus short cycle (measuring the performance of some in order to improve themselves) evaluation—and these are just the broad categories. Then we must ensure that it is valid (measures what it is supposed to measure) and reliable (consistent and reproducible).
The term “evaluation” is tossed around pretty freely in the performance improvement industry. Most organisations, if they bother to broach the subject, claim to provide ROI measurements of some sort for their training. We have noticed that the term evaluation is used rather loosely even in the academic literature. As it happens, while it may be a matter of semantics, the semantics matter.
“Evaluation” is sometimes confused or used interchangeably with words like measurement, assessment, validation and even Return on Investment (ROI). Evaluation determines the effectiveness and worth of the training (presuming that the metrics are properly set prior to the training)—meaning the effect of the training “in practice” and the accomplishment of defined objectives. As you will see below, at Huthwaite we have the capability to measure for four levels of evaluation. Measurement and assessment provide the objective data on which to base evaluation. They are themselves fascinating subjects. Validation is one level of evaluation which determines whether the training has delivered what it promised; ROI is another, far more complex, evaluation level that puts a dollar value on the gross benefit of the training. As you can imagine, the interchangeable use of these terms can lead to some marked confusion.
Evaluation Models
There is no shortage of evaluation models. Depending on the theorist, the models are either “broadly similar” or “no two are alike.” The most famous and—if there is such a thing—the industry standard is Donald Kirkpatrick’s four-level model, first published in 1959, but continually reworked to this day.
Huthwaite’s Definition of Evaluation
Huthwaite views evaluation (and most particularly, sales training evaluation) as a means of proving business outcomes by continuously improving individual trainee performance. We evaluate on four levels based on Kirkpatrick’s revolutionary model. We advocate these four levels because they are inclusive; they cover the field of evaluation. Each level has a useful purpose in relation to continuous improvement, but the breadth and depth that these levels provide offers wide flexibility in the correlation of training to results.
A Critique of Our Evaluation Model
Like all evaluation modelers, Kirkpatrick has his critics. So naturally we have had to meet and overcome certain criticisms of the model that reflect on our implementation of the model, such as: Kirkpatrick’s model represents a trainer’s notion of what constitutes effective evaluation not a business manager’s On the Results Level 4, ROI does not measure the worth or “soft” benefits like the impact training has on morale and employee commitment, or organisational adaptability to dynamic forces in the marketplace The accounting world does not yet view training as a capital investment, but rather as an expense because human capital has yet to be fully integrated into the Knowledge Age balance sheet.
A Measured Response
These are valid criticisms up to a point. But as it happens, none of them are insurmountable; each of these critiques can be mitigated with a well-conceived evaluation. For example:Our first step in every engagement is to gain senior management consensus by alignment of business goals with training; the business leaders understand from the beginning how the skill change will impact the bottom line. This tends to ameliorate any trainer-centric orientation.Huthwaite is a pioneer in putting “hard” measures to “soft” skills, and has been able to adapt the principles to value measurements of “soft” benefits. Based on our experience over thousands of engagements, increased skill level, which is quantifiable:
• Increases individual confidence which, in turn, boosts morale
• Increases sales which improves employee commitment
• Increases flexibility which enhances organisational adaptabilityWe can live with falling on the expense side of the ledger most of the time, for when pressed, at level four, we are able to show our value to the bottom line.
Conclusion
Sales training evaluation is a tricky business. It is a complicated subject fraught with misconceptions, misunderstandings, and frankly misleading claims in the sales training community. At Huthwaite, we take the position that evaluation properly understood is (a) not as difficult as all that, and (b) an absolute necessity—becoming ever more important to fully justify the vast amounts of money that are spent each year on sales training. But it must be done right. Through thousands of training engagements we have been able to refine and fine tune the genius of Kirkpatrick; we’ve taken the best of the best and made it accessible to sales management and business leadership. Below you will see our targets for the percentage of programs that ought to be evaluated at each level:
We are often asked whether and how we measure the results of our training engagements. The answer is entirely dependent on the three factors that drive evaluation design: (1) why we are doing the training? (2) why we are evaluating the training? and (3) who wants to know? Once having answered those questions we can say with confidence that yes, indeed, we evaluate our training and moreover that we do it to the specifications of the evaluation audience.
As you can see from the chart to the left, we evaluate almost every training engagement on Levels 1, 2 and 3; and we are quite prepared to evaluate our engagements on Level 4 as requested, with the stipulation that it costs substantially more, and ROI—which is very expensive—comes with a good deal of organisational political risk. Such is the nature of the beast. ROI evaluation, which many training organisations will claim to do, is both very expensive and hazardous. Be sure when navigating these turbulent waters that you are speaking the same language as your guide. Be clear on what you mean by evaluation and be clear on what your potential training partner means when they say, “Of course we will evaluate our training!” Caveat emptor.
Having your FREE evaluation with a business coach is a value of $500.
PDF
0 comments:
Post a Comment