Evaluating Training Programs LO15778

Lisa Carmichael (lscarmichael@earthlink.net)
Thu, 13 Nov 1997 01:22:11 -0800

Replying to LO15767 --

Replying to Brenda Schneider in LO LO15767 who wrote:

> Currently the organization does not have a way to evaluate the
> effectiveness of their training programs in the long run or at least 3
> to
> 6 months after the course has been presented. Does anyone have any
> suggestions or ideas of where I can look to learn about an evaluation
> system such as this?


I am also a graduate student who, believe it or not, just returned from a
lecture on evaluative research! Here are my suggestions for building an
evaluation process to measure the effectiveness of the training program
you are studying:

1. You didn't say in your note, but did the organization specify before
implementing the training program what they wanted the outcomes to be?
This is really essential to being able to effectively measure whether or
not the training has accomplished its objectives.

2. If so, what are they? (If not, measuring change will be difficult
because you didn't have an opportunity to collect data on these variables
before the training program was implemented.)

3. Assuming you know what are the goals of the training program, develop
a set of behaviors that illustrate the goals. In other words, what,
specifically, will it look like behaviorally when the goals of the
training program are met? Will the company receive fewer customer service
complaints? Will employees work among themselves to solve problems before
reporting them to a supervisor? Will turnover decline? (This is also
best done before the program is begun. However, if the behaviors are
identified now they can be used with some vigor to evaluate previous
training and even more effectively to evaluate subsequent training.)

You may want to consult a theory. For many change initiatives there are
models or theories which indicate the behaviors one can expect from
implementing the theory. You didn't say what your training program was
designed to address, but for sake of example, if I assume it is "job
enrichment," there is a theoretical model that spells out the dimensions
of behavior that should be impacted by a job enrichment program. You can
also develop a list of desired behavioral outcomes based upon your
experience, or that of the employees in the organization. What do they
think will change, what do they want to change as a result of the training
progam? This should be as specific and behavioral as possible.

4. Decide how to collect data on each of the behaviors/outcomes. If you
are looking at customer complaints, for example, telephone records or CSR
call logs are a good start. You may also want to survey those who have
customer contact to measure their perceptions.

5. Collect and analyze data to measure the identified variables. Did
something change?

6. To the extent your behavioral variables are good indicators of the
goals the training program was designed to meet, you can demonstrate that
the goals were or were not met by showing the extent to which the
behaviors are present following the training program.

The main things to remember are:

- identify as specifically as possible what you want to change as a
result of the training, and
- measure these indicators both before and after the training in order
to best determine change.

Good luck!

Lisa Carmichael


Lisa Carmichael <lscarmichael@earthlink.net>

Learning-org -- An Internet Dialog on Learning Organizations For info: <rkarash@karash.com> -or- <http://world.std.com/~lo/>