Our dilemma: How do we remove the barriers to knowledge about Klocwork’s toolset and developer best practices for creating high-quality code?

The answer: Klocwork Developer Network–a new online portal designed for learning, sharing and discussing all things source code analysis. We have had a lot of fun and a few sleepless nights as we assembled industry knowledge, online forums, computer-based training, best practices from industry experts, and lots of reference and learning resources.

A significant portion of the content on the Developer Network is open for public consumption. By registering and logging in, you get additional videos, demos, CBT and more.

We have a lot of fresh content to add to the site in the upcoming weeks and months, and we want to hear from you about what you would like to see. Why not register now at developer.klocwork.com? Then tell other Klocwork users about the portal too.

Visit Klocwork’s Developer Network at developer.klocwork.com.

Already a my.klocwork.com user? Access the Klocwork Developer Network using your existing my.klocwork.com login. (But note that my.klocwork.com remains the place to go for support tickets and for FTP access to the latest software releases.)

There was a recent article from Mark Pitchford titled: “Think static analysis cures all ills? Think again.” Obviously being biased working here at Klocwork, I take a major exception to what Mark has to say.

This article makes ridiculous claims. About the only thing Mark got right was that static analysis has been around for a long time. However it’s ludicrous to think that they’re the same as they were in the past. That’s like saying computers from decades ago are the same as today. The advancement has been huge for static analysis tools, especially in the last couple of years.

The author is really selling the merits of dynamic testing, which is great. Everyone should have the proper testing procedures in place. But static analysis is complementary–it’s another tool that will help you go through ALL the paths of your code to help you find bugs you’d otherwise miss.

One of the biggest reasons static analysis tools have taken off, in my opinion, is the level of integration. It’s quite simple to get results with static analysis tools with little effort. Especially compared with dynamic analysis tools.

Paul Anderson, a fellow competitor, sums it up very nicely in the comments. Check it out.

Static Analysis Tools Not Created Equal

Yes, it’s true (!) and as anyone in this space knows there is a huge difference between static analysis tools, their level of sophistication, and their approach to developer adoption. Gary McGraw & John Steven from Cigital describe their views on this topic including ‘5 pitfalls’ that customers should avoid when evaluating tools. These pitfalls mostly amount to the fact that analysis results across different tools, code bases, and tool operators can make results vary significantly, so be aware of this fact when conducting your benchmarking. Their overall recommendation:

“The upshot? Use your own code instead of a pre-fab evaluation suite. You probably have the makings of a good set of tests within your own organization’s application base….”

I agree with this and can honestly say we rarely, if ever, run into evaluations where customers exclusively use pre-fab test suites instead of their own code for many of the reasons outlined in their article. So, I’d say the market is (and has been for some time) embracing this recommendation wholeheartedly. So, beyond this recommendation, what else should customers consider when evaluating these tools? Here are a couple other significant areas to consider where you’ll find that, yes, all tools aren’t created equal.

• Environment support. In particular in the embedded software space, considerations such as integration with your build environments, compiler support, ability to work with multiple software branches, are all crucial considerations for a successful deployment. Not all tools have good support in these areas, but these capabilities can often make or break a deployment.
• Developer adoption. This is everything frankly, and a big part of achieving developer adoption is the quality of the analysis issues raised in the article. Obviously, a tool that generates accurate, useful results will get you well on your way to strong developer adoption, but that’s not everything. How are the defects described to developers, including the trace info? Do developers want to run their own desktop static analysis rather than fetching results periodically from the integration build? If so, how smart is the vendor’s desktop analysis?

So basically, picking a tool boils down to assessing: quality & flexibility of the analysis, support for your dev environment (not just the one you’re using in the eval!), and thinking ahead to developer adoption issues. Assess these three areas thoroughly and you’ll end-up picking the tool that’s right for your needs.

It happened again.  For what seems like the 100th time, someone reports to me that they are seeing a number of false positive reports on the resource leak checker.  For those not familiar with a resource leak, take a look at a previous post.  Although resource leaks apply across most languages, the place where this question keeps coming  up seems to always be in Java or C# code.  My last query came from Java code, so we will use that as an example.  Here was a report where the FileInputSteam is not closed on exit.

FileChannel sch = new FileInputStream(...);
FileChannel dch = new FileOutputStream(...);
try{
   ...
} catch (...) {
   ...
} finally {
   try {
      sch.close();
   }
}
...

Do you see this issue?  Clearly, they close the FileInputStream in the finally block.  The problem is that you can still throw an exception when you try to create a FileOutputStream.  The simple fix is to encompass the FileInputStream and FileOutputStream in the try/catch blocks.

Static analysis tools are great for finding bugs like this resource leak.  In this case, the report that a FileInputStream is not closed on exit wasn’t enough information to debug this.  This brings us to a very important discussion with static analysis tools–trace.

Gone are the days where a static analysis tool reports a bug at line X with no context.  Today you get detailed traces.  The trace contains conditions and assignments to give you clues about why the tool is identifying a bug.  In this example below, identifying the “source” and “sink” of any defect will help you trace through the start/end of the control flow.

More importantly in the case of Java, here we have a clear understanding where exceptions are thrown.  This would have helped tremendously for the example above.

Detailed traces are very important and can do many things for individual checkers.  So whatever you do, don’t just look at the bug description, pay attention to the trace!