The evaluation of e-learning looks at four levels: student reaction, knowledge transfer, behavioral change, and business results (Kirkpatrick, 1975). Critics of Kirkpatrick model say that it does not take the business impact far enough and that the final step in any training program should be a "fifth level" of evaluation -- financial return. This ultimate measurement controls the financial return on investment (ROI) of the training program.

Dependability

For a long time, organizations paid lip service to the human resource motto that "people are an organization's most important asset." Now senior executives have come to believe that employees and the intellectual capital they create can uniquely differentiate their company in the marketplace. Training expenditures now are viewed as critical investments in human capital, and an effective method of increasing employee retention.

Commensurate with this increase in training expenditures, senior executives are demanding more accountability from their training departments. In fact, 93 percent of training professionals surveyed at a 1996 conference said they are increasingly being asked to show the return-on-investment of their programs (National HRD Executive Survey, 1997). Training managers need to be able to answer direct questions about total costs, benefits, and bottom-line impact. Visionary training managers embrace cost-benefit analysis as a way to justify bigger budgets for technology and new training programs.

Cost-Benefit Analysis

Proponents of technology-based training have long touted its many benefits: reduction in learning time, increase in knowledge retention rates, cost savings. Brandon Hall, quoted in the August 1998 issue of HRMagazine, makes the generalization: "There's about a 50-percent reduction in time and cost over classroom training." (Roberts, 1998) The power of the cost-benefit analysis process is that it enables you to move from generalizations and assumptions to proof of the value of each and every program you develop.

Concepts of Cost-Benefit Analysis

At the simplest level, cost-benefit analysis answers the question "Was it worth the money?" In other words, what were the total costs to develop the program, and what were the total benefits realized? The following key concepts are factors in a cost-benefit analysis.

 Life of Training

        Every project needs to be measured across some time period. Technology-based training programs do not last forever. Their shelf life will be determined by things such as changes to content, changes in technology, and changes in business need. According to Hall's research conducted over the last ten years, most ROI studies show technology-based training is more expensive to develop and deliver over the short-term, but pays off over time. Predictably, three to five years of use is a conventional time period to apply for evaluating a training program.

 Alternative Delivery Options

        The most common method of showing the financial impact of technology-based training is to compare it against the costs for other forms of delivery. To come up with a comparison means, ask the question, "If we don't deliver the training via the Web, what would it cost for us to deliver it in a classroom setting?"

 Size of Users

        With technology-based training, the cost of development is not dramatically affected by the number of students utilizing it. However, the size of the target audience is extremely relevant when comparing the costs against instructor-led delivery. With live workshops, the number of students has a direct impact on expenses related to instructors, locations, and travel.

 Seat Time

        The total amount of time students will spend with the course is called seat time -- how long they will be in their seats. Seat time is always specified for instructor-led training, but is an estimate when given for self-paced, technology-based training. After all, a course that takes one student two hours to complete, might take another only 90 minutes. Increasingly, effective Web-based training is blurring the lines between instruction and just-in-time performance support. This factor makes estimates of seat time additionally tenuous.

 Estimated Revenue Impact

        Often the impact a training program has on sales and expenses is difficult to measure. In these cases, the impact on revenue is projected or extrapolated from known data. To do this, you would need to research costs associated with wasted materials, labor time for the manufacturing, identification of, and disposal. With this methodology, a defect-reduction number can be translated into a revenue-savings number.

 Opportunity Costs

        These costs are the lost revenues or increased costs associated with opportunities that will be missed because of the training program. This measure is increasingly being used in the competitive world of sales. Traditionally, for a sales representative spending time in training, a main measure of cost is the salary of the employee while in the training program. However, a more advanced analysis measures the opportunity cost of the user not being out in the field. According to Jim DeMaioribus, associate director of sales training at Knoll Pharmaceuticals, every day a representative spends in the field is worthapproximately $8,000 in revenue. Therefore a major advantage of technology-based sales training is its ability to maximize time inthe field and minimize the opportunity costs of sales training.

BACK