Tuesday 12 July 2011

How to create un-cheatable Software metrics?

This is the question, what I hear from time to time and of course, metrics are our data from our life supporting system, which makes this question interesting.

Unfortunately I think, the whole question stinks. Let's assume that you are a patient in a hospital after a really bad car accident. Your condition is that bad that you need a hospital life supporting system to survive. If doctors and technicians, who are maintaining it, goal is just get this data look good and if their bonuses were depending on it, then the target would not be your overall welbeing. How would you feel if you start to doubt that they are trying to cheating with this data?

If people are cheating with data from Software Metrics we are getting false information and it always leads to wrong decisions (usually Business kind of). Software Metrics like Running Tested Features, # of Defects, Velocity, Lead Time, Test Coverage, data from static code analyzers... you name it. It should give valuable information to the whole organization, how we are doing and the possibility to react when something starts to go wrong. Metrics should be for every stakeholder (developers, managers, business) and they have to be everybody's concern to keep our product alive and in good condition.

Usually when I have experienced such cheating, there is mostly a reward or punishment involved. If you are a Manager and still seeking to create a waterproof metric, I must remind you, that you have the most vicious opponents, Engineers, they will always find a way to cheat... if they want to.