I’ve talked about different metrics that are used to measure quality and the metrics that developers would use in practice.  But what about the tools themselves?  How are you measuring the value you are getting out of these tools?

In terms of static analysis, one obvious measurement is simply the bug fixes you have made.  Most organizations have a number they use to define the cost savings for each bug.  Using some research data from IBM puts the cost of fixing a bug before a release at 40-50 times cheaper.   Fixing a bug after release is the extreme and hopefully you’re finding these issues earlier and that’s where static analysis comes in.

I would think any static analysis tool offering would give you that simple graph.  Below is a couple of examples where you can track the fixes per component or per owner.

Bug fixes per compoent

Bug fixes per owner

One other thing that I think is very important is where you’re saving money.  When you run static analysis for example you have a couple of choices where you can find defects.  After the integration build or right at the developers desktop before they check-in their code.  Obviously if the developers are finding and fixing their issues while they code and before they even get checked in then it is saving even more money for the company.  Good static analysis tools will help you see the value of using it at the desktop.  A simple counter is all you need to see how many bugs were fixed in the code prior to check-in.  Immediate ROI from using static analysis.

As the topic suggests, this kind of argument has been around for some time.  Most developers can recognize the need for tools but once you start breaking the developer’s day-to -day workflow you might as well flush that tool down the drain.

What developers need is a tool that seamlessly integrates with their development environment and their workflow, so they can meet their quality goals without taking a big productivity hit.

It’s one thing to provide plug-in tools for the more popular IDEs like Visual Studio and Eclipse, but it’s an added bonus when defect detection is a seamless part of the edit cycle. No buttons to click, just continuous analysis and issue highlighting while you work.

Let’s take the analogy to the spell checker.  Initially, you had to click a button to spell check your document. That has obviously changed dramatically. Now we see any mistakes we make as we type them (and can even fix them automatically).

That’s what we were thinking when we introduced continuous analysis in our plug-in tools and our source viewer for command-line tools, Klocwork Desktop.

Here’s the spell checker equivalent for source code analysis:

The above screenshot is from our Visual Studio plug-in.

When you open or save a file, the analysis runs in the background. A bug marker in the left gutter and a squiggly line, in the true spirit of the spell checker, clearly marks the detected issue.

Find ‘em and fix’em while you work.

In the spirit of the FIFA 2010 World Cup, I thought it would be fitting to describe how software developers can relate to the game.

1. Announcers – Have you ever really listened to what the announcers say?  One of my favorite things to listen to is the very opinionated soccer announcers.  Some of the things they say just make me laugh.  For example, when the announcer was describing the uncertainty of the game – “There’s one thing for certain, there is no score.”  or in this year’s World Cup describing a slow and boring game – “It’s like they are playing in slow motion”.  I’m not saying developers are opinionated…no way ;).  One thing that is similar is the comments developers will put in the code.  One of my favorites:

//When I wrote this, only God and I understood what I was doing
//Now, God only knows

For more funny comments go here.

2. Money – Soccer players do what they love for vast amounts of money.  Developers do what they love…well okay maybe not the second part.




3. Vuvuzelas – whether you like them or not you are stuck with listening to hundreds of Vuvuzelas playing their merry tune.  Despite all the complaints it will continue to haunt spectators until the tournament ends.  So why is everyone blowing the god forsaken plastic tubes?  Well my first guess is that they are drunk, but I think mostly because it is fun.  So as a software developer you don’t get to blow on the vuvuzela but I bet you would want to when you finished your work or the latest complicated feature?  Hopefully this is not because you’re currently drunk.




4. The thumbs up – In a meeting I had with our customer advisory board there was one individual who kept giving the thumbs up.  I understood that he was voicing his agreement with what we were talking about but I never understood why it was always with the thumbs up…until the World Cup started.  Seems to be the universal sign for the soccer players to say “nice ball” or “good play”.




5. Drama – Have you ever noticed how the majority of soccer players act when they have been fouled?  They dive 10 feet in the air, roll 16 times and clutch their chest like they were just shot.  Okay maybe I’m exaggerating a little but the point here is that some of these players are under the impression that they may get nominated for the next Academy award.  How does this relate to the software developer?  Well think of the code review, who really likes hearing that their baby is ugly?

A while back, I talked about how I keep running into organizations that seem to go out of their way to make developers’ lives hell.  I’ve run into several examples where developers had to switch between different environments just to write and compile code.  That’s as productive as watching paint dry and as much fun as rearranging the deck chairs on the Titanic.

For teams that want to run source code analysis in these types of environments (or any kind of dev tooling, frankly)  it’s very difficult for vendors to support.  I did my usual PM grumbling about these environments but since that post exactly 1 year ago I’ve come to realize that these environments are a reality and we need to figure out a way to support them.  Maybe it’s not productive but organizations are making it work.  I’m sure they would even argue that they have made it productive (good luck to them).  It’s for this reason that Klocwork has given in and instead of pointing our finger and making fun (I swear I never did), we’ve decided that it’s in our best interest to make sure we provide these customers with the capability to run static analysis.

A couple of releases ago, Klocwork introduced a new tool called Klocwork Desktop that provides Klocwork command line users with the same graphical capabilities that one would get from Visual Studio or Eclipse.  This tool was great for users who never used an IDE.  With Klocwork’s 9.1 release we have extended Klocwork Desktop’s reach by providing a remote capability that’s designed to support the type of environments described above.  Using Klocwork Desktop in remote mode allows users to view their source and detected-issue information when Klocwork Desktop does not have direct access to source files or defects, yet still get the benefits of finding and fixing your defects before you check-in your code.

One really cool feature that is part of this is the “you’ve got mail” notification.  At first, I have to admit this is something that worried me.  If I had to label one thing as a productivity drain it was those annoying alerts you get of new email coming in.  Of course right in the middle of doing something important you get distracted by a new email with plans for the next party (or in my case hearing about the kids latest poop explosion).  The first thing I always do is turn it off.  But in the case of finding bugs while coding, it only makes sense to give you these notifications in a heartbeat.  So you can actually be writing code on some machine in Jakarta and automatically your machine in San Jose is alerting you of bugs.  Pretty neat stuff.

Previously I posted the value of using coding standards, specifically MISRA C and MISRA C++.  This time I wanted to go through some general experiences we had with some of the checkers, specifically the ones that seem to throw a lot of violated rules, to the point that on some code bases MISRA flagged more than one error per LOC!

There are still tons of great rules you can apply even if you don’t make an embedded product.  But as I said before, it doesn’t make sense to turn on all the MISRA rules.  After running through many code bases and looking at the value of MISRA we certainly noticed a trend with a few culprits.  Here are a few examples that we found to be noisy with non-MISRA compliant code.

MISRA C 6.3 and MISRA C++ 3-9-2:

MISRA has the distinction of “required” rules and “advisory” rules.  This is an “advisory” rule.  Essentially it wants you to avoid using types like char, int, short, long etc. and to use specific length typedefs instead.  Obviously, many code bases use the basic types, so be prepared for many issues.

MISRA C 14.9 and MISRA C++ 6-4-1:

This is a “required” rule.  This rule  is really about making sure you have braces for your if/else keywords.  Good practice to have but how many of us really do this?

MISRA C 12.13 and MISRA C++ 5-2-10:

This is an “advisory” rule.  The rule doesn’t want you to mix increment and decrement operators into an expression.  This makes sense because it can be pretty confusing to read.  But in our experiments this seems to be something that many developers do.

MISRA C 17.4 and MISRA C++ 5-0-15:

This is a “required” rule.  The rule only wants to allow you to use array indexing for pointer arithmetic.  Everything else is non-compliant.

MISRA C 14.7 and MISRA C++ 6-6-5:

This is a “required” rule and another control flow example.  A function can only have one point of exit at the end of a function.  I can understand this but as you know, that is not reality.

MISRA C 13.2:

This is an “advisory” rule.  It states that tests against zero should be made explicit.  In other words: if (x != 0) is the proper way, not if (x).  The exception to this is if the operand is Boolean.  I don’t know about you, but you can crown me the super wiener on this? I never make it explicit.

So if you plan to pick up MISRA on your existing project beware of these rules.  I’d like to hear if you do any of those things in your code base.

In a previous post I discussed the process where we practice dogfooding.  This is the process of using Klocwork on Klocwork (KonK).  We started this program several years back with the hopes that we would learn some valuable lessons about usability, performance and anything else that would give us an edge.  The truth is that KonK has consistently allowed us to test our design assumptions early by allowing our own developers to use Klocwork as part of their development.

One of the unexpected results was inadvertently uncovering data that further validated for us the importance of bugs found by a static analysis tool. We correlated testing-found issues and KonK (static analysis) issues assigned to each developer. The result as this graph shows (note: graph is using a logarithmic scale and is a few years old), is there was a very high relationship between the two. Those familiar with bugs found by static analysis tools know that they are often quite different in nature from a bug found by testing where you’re testing requirements, yet developers with a high count of testing-found issues also had a high KonK count. In some cases they pointed to the same problem, in others an indication of the overall bad to the bone problems in some of the new components.

Since we already eat our own dogfood, we’re already believers in the use of static analysis, but this kind of data is a great example of the benefit of finding issues early using static analysis

Dogfooding?  Is this the process of making sure Rover is well fed?  Maybe it’s a movement of people eating dog food?  Or maybe Rover IS the dinner (cue animal activists).  No, dogfooding is coined from the saying “Eating one’s own dog food”.

So what on earth am I talking about.  Well first I’m breaking a golden rule here when it comes to blogging which is talking about your company (I don’t know how I’ll sleep tonight).  Klocwork does eat its own dog food.  We call this KonK – Klocwork on Klocwork.

So why is this important?  Ultimately we make a software product that we sell to other companies that make software.  So who better to experience first hand what we are designing.  By using KonK and updating it frequently it gives us immediate feedback on usability and scalability (our code base is quite large).  Plus being in the business of bug detection it helps us sort out the value of the quality of those bugs.  As the product manager I’m not using it day to day like the developers, so they are the ones that bring any kind of deficiencies to the design front and center.  Hopefully I can talk a little about the benefits and conclusions we have made by using KonK in a later post.

One thing that has helped me with dogfooding is requirements capture.  Being in product management obviously the clear objective is to work closely with customers to define requirements and your product direction.  Those requirements don’t necessarily paint the picture as much as you would expect or hope.  Now that I can play with the intended design on our own code base it paints a clear picture of what may be missing or what may just be plain ugly.

What is MISRA? More Irrelevant Software Requirements Again…uh no but certainly the sentiment of many developers.  MISRA (Motor Industry Software Reliability Association) is a coding standard, which first released MISRA C in 1998 and has since been revised.  Obviously, this came out of the automotive sector with a clear focus on helping software systems to be more reliable and maintainable.

MISRA has since grown.  Now you see more and more industries adopting these standards.   In 2008, MISRA released the C++ equivalent standard.  So the obvious question is, should I apply this to my software source code base?  In short, yes.  Maybe you don’t need ALL of the MISRA rules, but there are certainly some that are worth your time.

Examples range from the very simple checker rules sets that mostly fall into the coding style category:

MISRA C rule 19.1:  All #include directives in a file shall only be preceded by other preprocessor directives or comments.

MISRA C rule 2.2: You should only use /*…*/ style comments.

Down to more important issues (I hope everyone has the tools in place to find this):

MISRA C rule 9.1: Unintialized variables.

It really depends what you’re trying to accomplish.  Certain rules such as 9.1 are very important, while others can help you with portability or just serve to make your code more maintainable and less complex going forward.

Word of warning, taking an existing software code base and apply the entire MISRA rule set is guaranteed to throw many errors.  Take the zlib project, obviously not MISRA compliant, but to illustrate the impact of turning on all the MISRA rules, will throw over 5000+ issues.  That is an issue for every line of code…yeah that’s useful.  So be warned, don’t try to boil the ocean.  Start iteratively, even if that means only one rule at a time.

In my previous post I discussed where I thought Agile really falls flat.  The problem I have is working remotely.  Several times now I’ve misinterpreted what exactly we covered in remote meetings.  These have been mostly minor things but they do add up.

But here is where there is just a massive difference between Waterfall versus Agile.  By far the biggest lesson for me and why I love Agile is all based on visibility.  Having a working product in one simple iteration means the world.  So even though I was ranting in my previous post, imagine if we were doing waterfall.  Chaos could have ensued once the product was integrated (maybe months after).  As many point out, iterative development shows us up front (in two weeks or less) that we have a major problem.  I’ll stick to Agile, thank you.

So Agile is great, I’m all aboard but I still have to do something about working remotely.  Once I get this addressed I should be sailing.

I’ve come across a few things that people are doing .  Mostly with some interesting video conferencing techniques (here and here).  Do these really work?  There are some studies which are interesting but they are not really applied in an Agile context.  I did come across one article that might be helpful…although I need to shell out a few bucks.  I did find one reference on martinfowler.com that I thought was useful as it goes through the key points you need to consider with offshore development.

Overall, I still don’t have a good plan to deal with this.  So let me ask everyone out there, please let me know if you know of any good references.  I could certainly use some help.

Unlike Todd who is this blog’s main Agile expert, I’m pretty new to agile.  I’ve gone through the typical training (CSPO) and all the other good stuff,  so I’m drinking the Kool-aid.  But I thought I would provide my perspective,  now that I’ve been working in an Agile shop for a while and tell you what I think really sucks.  I’ve read lots of warnings why Agile can fail and I’ve tried to stay focused on overcoming the hurdles.

Being a product manager, one of the things that is really ringing true to me is where Agile falls flat–working remotely.  Lots of discussions on how much it can suck here, here, here and here.  As part of my job, I travel… lots.  If I’m not at a customer meeting I’m at the next trade show.  This means many design and planning meetings are done remotely.  I end up finding we have developed something different than what was originally discussed or at least what I thought.

This gets frustrating because it has a major impact — all of a sudden as opposed to having that working functionality at the end of the iteration, you are set back by redoing that same work the very next iteration.  About as productive as watching every single sport of the Winter Olympics (I don’t even like figure skating…I swear).

As much as I’m puking all over Agile, I’m still very invested (messy as it is).  Let’s face it, offshore development is a way of life and there are many things people are using to work in this remote context.  Next week I’ll post the opposite side of things and discuss the consequences of working remotely along with the value Agile does give you.