A Learning Process - Knowledge Management Metrics LO21878

Mark W. McElroy (mmcelroy@vermontel.com)
Wed, 09 Jun 1999 18:23:08 -0400

Replying to LO21867 --

lsmith@worldbank.org wrote:

> For all the interest and money spent on KM there seems to be relatively
> few attempts to actually quantify the impact and results in business
> terms. Most Metrics I've seen are more 'web metrics' than KM metrics, in
> that they count page hits etc. but don't have any way to consider the
> actual impact of the use of the knowledge.
> Anybody got any ideas?
> Enclosed below is a short piece I wrote to clarify my ideas, it is a
> work in progres, but I'd love feedback and dialogue.

Great topic! I've thought a lot about it and have also read your piece.
My first reaction was that it mixes metrics with methodlogy and ROI to the
point where I felt the intended discussion of metrics suffered. Metrics,
methodology and ROI are clearly closely related, but a discussion of just
metrics should, IMHO, stick to just, well, metrics.

Let me get your reaction to my own thoughts on KM metrics. First, as I
have said before on this list, I think the most useful approach to KM is
to think of it as an implementation strategy for organizational learning.
Second, as a strong proponent of the complexity-theory brand of KM, I see
knowledge as nothing more than rule sets held by individuals and the
groups to which they belong. These rule sets evolve over time, and the
rejection of old rules (knowledge) in favor of new rules (new knowledge)
is what we refer to simply as "learning."

This natural process of knowledge production has been mapped out by the
Knowledge Management Consortium (KMC) and can serve as a useful framework
against which metrics can be applied. "Successful" KM initiatives would,
therefore, be ones that make this natural knowledge production system work
better. The telltale sign of a better working system, in turn, would be
a) A more complete knowledge production system has been established where
previously, perhaps, there was only part of one, and b) the turnover and
distribution of rules and rule sets have been improved. The second area,
in particular, offers what I feel is a very promising metric for measuring
the relative success of KM efforts.

Note that I am not talking about ROI here. Rather, I am suggesting that
if a well-oiled knowledge production system is operating up to its full
potential, one would expect to see more turnover and greater penetration
of rules (knowledge) throughout the enterprise--that is, higher rates of
shared learning. Using rule mapping tools and techniques, measurements of
rule set "cognigraphics" could be taken on a before-and-after basis. If
rule making processes have, in fact, improved, this metric would
empirically capture the evidence by measuring the presence of a) more
rules in use by the organization, b) more NEW rules in use, and c) more
rules commonly held by disparate parts of the organization. The point is,
rules are measurable. If you can equate knowledge with rules and rule
sets, and if you are able, as I am, to think of KM as something that's
applied to natural knowledge production processes, then the effects of
that application can be conclusively measured: either more new rules are
being produced by the system or they're not. That's your metric!

I found the following remarks of yours to be close to what I'm talking
about here:

> "Indeed an interesting Metric would be to measure the volatility or
> speed of knowledge. At what speed do best practices become outdated? What
> is the 'freshness' of knowledge? How is this changing?
> Some projects will produce true innovations. Not incremental
> improvements in existing best practices, but radically new concepts, ideas
> or products. These in turn will be fed back into the beginning of the
> business cycle as new business drivers or imperatives which in their turn
> will demand a whole new cycle of KM support, with different points of
> value and new metrics."

If you'd like to see the KMC's model I spoke of, let me knowl and I'll
send it to you separately.

Mark W. McElroy


"Mark W. McElroy" <mmcelroy@vermontel.com>

Learning-org -- Hosted by Rick Karash <rkarash@karash.com> Public Dialog on Learning Organizations -- <http://www.learning-org.com>