Time and time again I get asked, how does static analysis fit into my existing bug tracking system? ”I need an integration with my system (i.e. Bugzilla) because that is what we use everyday. Every time I find a bug I need to track this through my system.”

This is where I take a deep breath as I scream on the inside. Taking every bug and putting that into your bug tracking system is just wrong. Horribly wrong.

The best way I can describe this is through the compiler analogy. Every time you add a feature or bug fix, you run your compiler. Your compiler complains, saying you have a syntax error and you just fix that, rinse and repeat.  Would you ever send your syntax errors to your bug tracking system? Uh, no. So why do the same for static analysis? Just because it is a critical buffer overflow or memory leak? No.

Why impose any kind of workflow for the developer when it comes to bugs? There should only be one thing the developer has in mind. Either fix the issue or ignore it. Just like they do for any syntax error from the compiler.

This is really a discussion of your usage of your static analysis (or source code analysis) tool.  What I’ve described is the process where the developer is using static analysis at the desktop, in their favorite text editor or IDE. Of course, there is an exception to this. You can use static analysis on the full system run only. I highly discourage this for various reasons but I’ll save that for another discussion.

The chances are that you could have a product that is already established, so you probably have a backlog of issues.  Static analysis can certainly be used to triage these bugs and output them to your bug tracking system.

I hope my message is clear, if you use static analysis correctly (allowing the developers to fix bugs as they create them) and a bug appears, just plain fix it.  Yes, it’s that simple.

As we continue to rollout Klocwork Insight 9.5 our message around making static analysis simple is taking hold. To put the change we’ve made into its appropriate context, let’s think about how spell checkers mainstreamed, and how a somewhat obvious (looking back) usability change turned this amazingly useful technology from something you do at the end of writing a document, to an activity that just automatically works while you create documents, making people more productive in the process.

Two Different Spell Checker Usability Models

On the surface the difference looks subtle but the changes are huge. The first scenario required the user to take action, interrupt their workflow, and was a “separate” activity from writing a document. The usability model we’re all familiar with now is naturally incorporated into a writer’s workflow. It still does the same thing: it checks your spelling against its dictionary, provides suggestions, and will auto-correct for you. But the difference in ease-of-use and end-user adoption of the technology has obviously been huge.

That’s what we’ve done with static analysis. We’ve taken a complex analysis technology that used to only live at integration build time, moved it down to the desktop in 2008 (much like the usability model on the left above), and now with Klocwork Insight 9.5 have introduced the model to the right. Here are some links to short Klocwork Desktop Analysis for Visual Studio demos showing this capability in-action.

Static analysis captures the current state of your code base and helps you answer key questions about the quality, security and maintainability of your software project.

Think Magic 8 Ball with build omniscience and powerful reporting tools. OK, maybe Magic 8 Ball isn’t a good analogy.

Answers to what questions, you ask? One we often hear from customers is: Where do I start?

A good place to start is a report that captures the distribution of defect types from your current build.  For example, we recommend that our customers glance over the Top 10 Issues report in our web-based build reporting tool, Klocwork Review while indulging in their morning cup of coffee:

Magic 8 Ball can't do this. Here's a defect distrubtion view of your build.

With this build snapshot and your caffeine jolt,  you can quickly identify defects of interest to your organization, such as null pointer dereferences and memory leaks. If you wish, you can set up filters (we call views) to show only these defect types in your report.

Your next step is to get your developers using static analysis on their desktops to prevent the injection of these high-priority defects into the build in the first place.

Once a policy of pre-checkin static analysis usage is put in place, pay attention to new defects injected into the build from that point on. If you see a spike in new defects, then investigate.

The magnitude of that y-axis is not what matters most; it’s the overall trend that counts.

For my next post, I’ll take a look at reports that track your cost of ownership and show you what success looks like.

The Code Integrity folks have developed a lot of best practices on deploying static analysis and have compiled many of them in a solid whitepaper. They include a Top 10 list of what they call “The Well Traveled Path to Success”. Below is their (somewhat paraphrased in spots) list.

1. Determine who cares. Who has a vested interest that bugs actually get fixed. How much do they care?

2. Get an expert to tune the solution for your codebase. Static analysis tuning will maximize defect finding while minimizing false positives.
3. If possible, pilot with a small group to gain early successes.
4. Appoint the proper roles, particularly management sponsor, administrator, defect triagers, fixers and verifiers.
5. Set up the proper process, incentives and consequences. Integrate the SCA tool into your environment. Automate and simplify as much as possible.
6. Get a team to handpick good, high-priority defects for the team rather than have them sift through potential false positives.
7. Set up a central resource website that includes simplified documentation, policies, procedures, roles, reports, etc.
8. Set up various reports like the daily dashboard, top ten list and the “wall of shame”. Make it public. Do a little bit of marketing.
9. Train and mentor the team providing guidance, support and discipline. Either in-person or static analysis e-learning courses work.
10. Determine success criteria and measure it. Provide status updates often, work on a source code analysis ROI model that works for your organization.

I agree with the general thrust of most of these, but some might be overkill depending on the size of your deployment. My other quibble is that many of the recommendations presume a centralized defect triage model where you’d have a central group of code reviewers sifting though bug reports.

That’s a common deployment model, but we’re seeing more people choose to just provide the tool to their developers via desktop static analysis. With the possible exception of your backlog, this will eliminate (or greatly reduce) the need for a central code review team that stares at bugs all day long. Regardless, they’re all good considerations to at least, well… consider.

With the launch of the Klocwork Developer Network, we’re making a deliberate and concerted effort to make many of these kinds of deployment resources freely available to our customers. I’ve included links where appropriate.