I’ve talked about different metrics that are used to measure quality and the metrics that developers would use in practice.  But what about the tools themselves?  How are you measuring the value you are getting out of these tools?

In terms of static analysis, one obvious measurement is simply the bug fixes you have made.  Most organizations have a number they use to define the cost savings for each bug.  Using some research data from IBM puts the cost of fixing a bug before a release at 40-50 times cheaper.   Fixing a bug after release is the extreme and hopefully you’re finding these issues earlier and that’s where static analysis comes in.

I would think any static analysis tool offering would give you that simple graph.  Below is a couple of examples where you can track the fixes per component or per owner.

Bug fixes per compoent

Bug fixes per owner

One other thing that I think is very important is where you’re saving money.  When you run static analysis for example you have a couple of choices where you can find defects.  After the integration build or right at the developers desktop before they check-in their code.  Obviously if the developers are finding and fixing their issues while they code and before they even get checked in then it is saving even more money for the company.  Good static analysis tools will help you see the value of using it at the desktop.  A simple counter is all you need to see how many bugs were fixed in the code prior to check-in.  Immediate ROI from using static analysis.

In a previous post, I talked about different types of metrics and how they can apply.  These were mostly focused on a system level and provided insight into system complexity from  release to release or build after build.  Now, let’s take a look at some other metrics and how they can be used effectively with the developers.

There are literally hundreds of metrics you can measure, so what is really important?  That’s a tough question because it depends on your organization.  Perhaps you have some coding standards in place or simply want to measure some basic functionality to help developers improve code quality.

Here are some of the more common metrics I’ve seen used in practice:
•    Lines of code
•    Number of functions/methods
•    Commented code
•    Nested includes
•    Number of compiler directives

Most of the metrics listed above are about maintainability.  The more lines of code you have or the more of functions/methods, the more you have to maintain for an individual file.  If these values are high, then perhaps it deserves a new file or new function.  The opposite end is commented code, where you obviously want more code commented.  Nested includes are a great way to alert the developer that adding a new include has all of a sudden added a long compile tool chain, potentially increasing your compile time and size of your system.  Tracking compiler directives is something, I think, is really useful–especially with platform specific code.  Compiler directives are a great way to give you visibility on how much you are using conditional compilation and hopefully, gives you a good understanding of the scope if a new platform is added.

Thresholds are the way to use any of these metrics.  For example, create a threshold that would throw an error if the number of functions per file is greater than 100.  Or,  first give a warning to developers when the number of functions exceeds 75 per file and then, if it goes over 100 print an error.  But what values should you use?  You can use the standard deviation to figure this out or use coding standard guidelines.  For example, many standards say that a method should not reach over 1000 lines or code, or the maximum level of include nesting should not exceed 8.

If you find that these metrics are not enough, start combining them.  Any good tool worth its  salt will let you customize these core metrics, so you can start creating your own calculations.  One example is using the lines of code metric with the commented code metric to create a calculation on the percent of comments.  We are all guilty of not documenting our code.  Use percentage of comments to make sure everyone knows when they are under the norm,  so if you have less than 10% of your file commented you get an error and less than 20% you get a warning.

Showing developers potential areas of concern will help them to be smarter at what they do and spend more time focusing on what they are good at–innovating.