Do Measurements Matter? (does a watched pot boil?)


There’s long been a debate about the role of metrics in software development organizations. There was the old adage that went “If its important, then its important enough to measure.” More recently, one of the proponents of that mantra, Tom DeMarco has questioned his line of thinking, observing that “The book for me is a curious combination of generally true things written on every page but combined into an overall message that’s wrong” This can be found in its full glory at http://www2.computer.org/cms/Computer.org/ComputingNow/homepage/2009/0709/rW_SO_Viewpoints.pdf. It presents some interesting and very thoughtful perspectives.

I recently ran into another interesting article, referenced from the freakonomics blog, pointing to a (very loosely) related study of measurement and accountability in schools in England. In this article “A natural experiment in school accountability: the impact of school performance information on pupil progress and sorting”, (http://www.bristol.ac.uk/cmpo/publications/papers/2010/wp246.pdf) the authors observed that the ceasing of measurements and rankings of schools in Wales had a profound effect on the school systems’ effectiveness.

So, like many things in life, there are multiple sides to a story. In this case, I suspect that the right answer (at least as pertains to software organizations) is measure what’s important, and on a macro scale, as a sort of score card as a means to set goals. But maybe don’t get so carried away measuring the details that one completely redirects behaviors towards miniscule observations and end up with a badly unintended corollary to  Heisenberg (that measuring some things has unintended consequences rendering related measurements useless).

Perhaps the takeaway is that its best to hold groups accountable, but only to those things that truly matter.

Advertisements

About Mike Rodbell

I'm a technology leader, engaged in developing software for the telecom, online commerce, and business process/analysis markets. All of the teams I've worked with have had a great deal in common. They need to be good at what they do, listen, share, and collaborate towards a shared set of goals. This blog is dedicated to those activities. I hope you enjoy reading it.
This entry was posted in Change and Behavior, management, Metrics, software development and tagged , , , . Bookmark the permalink.

2 Responses to Do Measurements Matter? (does a watched pot boil?)

  1. Mike says:

    This is major coming from Tom. I went to a seminar of his about 35 years ago, no, really, and he definitely had the religious fervor then. He had some interesting ideas. He could be a bit over the top in his younger days. Maybe he’s mellowed. It’s possible.

  2. Mike Rodbell says:

    Hi Mike,

    Yeah, when I first heard this one (its about a year or so ago. The pointer came from an employee of mine who was not a big fan of metrics), I was really surprised. I hear people mellow. Word has it, I may have….

    Mike

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s