In today's business environment it is becoming increasingly important to measure the effectiveness of training programs. Training must compete with other business initiatives for funding. Training departments must ensure that the most effective programs are funded and the least effective programs are dropped. C3's CompleteROI™ system assists organizations in measuring and reporting on the effectiveness and results of training programs.

Training program evaluation consists of several different components (Hodges, 1999):

Components collectively referred to as the summative evaluation are highlighted in blue. The CompleteROI system is intended to assist organizations in performing a summative evaluation of their training programs. Components of a summative evaluation include:

Reaction evaluation: The reaction evaluation takes place soon after the training and measures participants' satisfaction with the program. At its very simplest, the reaction evaluation can be a straightforward evaluation of how much students liked the program. But the reaction evaluation has significantly more value if more detailed questions are asked. These questions can include:

• How relevant the course material and the objectives of the program were perceived to be to the participants' work.
• How transferable the course material was perceived to be to the workplace
• How well the course was able to maintain interest
• How effective group activities or interactions were perceived

Reaction evaluation is typically performed using surveys, either online or using paper-and-pencil format, at the completion of a program or at the end of individual segments. Interviews and focus groups can also be helpful. Reaction evaluation is referred to as "Level 1" in Jack Phillips' ROI model.

CompleteROI Reaction Evaluations are a component of the CompleteROI System.

Learning evaluation: The learning evaluation is intended to test how well participants have learned the material taught in the course. The evaluation consists of a post-test, or pre- and post-tests, with questions tied to the specific objectives of the course. This measure assesses how much student's understanding of the material taught in the course has improved due to the course. Learning evaluation is referred to as "Level 2" in Jack Phillips' ROI model.

C3 offers CompleteROI Learning Evaluations as a component of the CompleteROI System.

Performance evaluation: Conducted as much as three to six months after the completion of a program (and in some cases before the program as well), the performance/behavior evaluation measures how well participants have transferred the program learning into into tangible, observable on-the-job performance.

A performance/behavior evaluation in many cases is a 360 degree review of individuals in areas tied to the objectives of a course. Behavioral scorecards/observational surveys are used to measure others' impressions of the participants' performance in course-related areas. A participant can be evaluated by his or her supervisors, direct reports, peers, customers, and others.

Performance/behavior evaluations can take place after a course has completed, or they can be run both before and after a course to best measure differences in performance. CompleteROI Performance Evaluations are a part of the CompleteROI System.

Impact evaluation: The business impact (and potentially the financial return) of a training program is measured through an impact evaluation. Metrics related to the training program are tracked over the course of the program. These can include increases in output, quality improvements, decreases in defect reports, and participants' own impressions of training's contributions to business goals. To measure ROI, these metrics are tied to financial drivers.

It is difficult to truly isolate the ROI and financial impact of training programs, as they do not take place in a vacuum. For this reason, it is important to be extremely conservative in the calculation of ROI figures and ensure that more benefit is not attributed to training than is deserved. ROI can serve as an extremely useful metric to compare investments in training to investments in other corporate projects.

C3 offers the capability to tie output measures to financial drivers in its CompleteROI Impact Evaluations, part of the CompleteROI System.

Pricing depends on desired components and usage. Please contact an account representative for a quote for your organization.