Time and time again I get asked, how does static analysis fit into my existing bug tracking system? ”I need an integration with my system (i.e. Bugzilla) because that is what we use everyday. Every time I find a bug I need to track this through my system.”

This is where I take a deep breath as I scream on the inside. Taking every bug and putting that into your bug tracking system is just wrong. Horribly wrong.

The best way I can describe this is through the compiler analogy. Every time you add a feature or bug fix, you run your compiler. Your compiler complains, saying you have a syntax error and you just fix that, rinse and repeat.  Would you ever send your syntax errors to your bug tracking system? Uh, no. So why do the same for static analysis? Just because it is a critical buffer overflow or memory leak? No.

Why impose any kind of workflow for the developer when it comes to bugs? There should only be one thing the developer has in mind. Either fix the issue or ignore it. Just like they do for any syntax error from the compiler.

This is really a discussion of your usage of your static analysis (or source code analysis) tool.  What I’ve described is the process where the developer is using static analysis at the desktop, in their favorite text editor or IDE. Of course, there is an exception to this. You can use static analysis on the full system run only. I highly discourage this for various reasons but I’ll save that for another discussion.

The chances are that you could have a product that is already established, so you probably have a backlog of issues.  Static analysis can certainly be used to triage these bugs and output them to your bug tracking system.

I hope my message is clear, if you use static analysis correctly (allowing the developers to fix bugs as they create them) and a bug appears, just plain fix it.  Yes, it’s that simple.

As we continue to rollout Klocwork Insight 9.5 our message around making static analysis simple is taking hold. To put the change we’ve made into its appropriate context, let’s think about how spell checkers mainstreamed, and how a somewhat obvious (looking back) usability change turned this amazingly useful technology from something you do at the end of writing a document, to an activity that just automatically works while you create documents, making people more productive in the process.

Two Different Spell Checker Usability Models

On the surface the difference looks subtle but the changes are huge. The first scenario required the user to take action, interrupt their workflow, and was a “separate” activity from writing a document. The usability model we’re all familiar with now is naturally incorporated into a writer’s workflow. It still does the same thing: it checks your spelling against its dictionary, provides suggestions, and will auto-correct for you. But the difference in ease-of-use and end-user adoption of the technology has obviously been huge.

That’s what we’ve done with static analysis. We’ve taken a complex analysis technology that used to only live at integration build time, moved it down to the desktop in 2008 (much like the usability model on the left above), and now with Klocwork Insight 9.5 have introduced the model to the right. Here are some links to short Klocwork Desktop Analysis for Visual Studio demos showing this capability in-action.

In this continuing story about the journey to source code awesomeness, we left off at the point where we identified priority defect types for your organization, kicked off pre-checkin static analysis on developer desktops and saw build-over-build improvements in our trending reports as a result.

The next question we tackle here is: What is my cost of ownership?

The answer, my friend, is not blowing in the wind, it’s in your Complexity Trend report:

Complexity Trend report

Why there? you might ask. Well, it’s because there’s a straight-line correlation between the complexity of a function and its cost per line to code and test.

If we see an increase in the Complexity Trend, we can use Complexity Details to pinpoint modules or files containing the most complex methods:

Complexity Details report

Looking at this report, I can see where my ownership cost is localized. More often than not, we find that the modules that contain the most defects are also those that contain the highest number of  complex methods.

Klocwork Insight metrics use McCabe Cyclomatic Complexity to assess the complexity of a program.

Another indicator of the high cost of ownership is the point at which defects are caught and fixed. Finding and removing bugs as early as possible in the development cycle lowers your ownership costs. This report shows fix activity between builds. You can toggle between views showing fix activity in the integration build and on the desktop. For now, we’ll look at All fix activity:

Fix activity by module

Here we see a spike in activity that occurred during the start of a new feature release.  If the upward trend continues, it can indicate a couple of scenarios. Either:

• Your development team is growing and more developers are using static analysis on the desktop, or
• Your team size is the same but developers are using static analysis more consistently.

Either way, more activity means more value from the tools and a lower cost of ownership. Developers fixing defects before check-in lowers your cost of ownership considerably.

Any more questions?

As any static analysis or source code analysis vendor will tell you, false positives are a way of life.  As any user will tell you, false positives suck!  So what do you do about them?  Make the tools better at finding the real issues and provide automated filtering capabilities.  But I’m not here to talk about false positives where the tool is utterly wrong.  What I want to talk about today is what I call “perceptual false positives”.

I’ve had discussions with customers where they tell me 80% of all their defects are false.  Odd, I know static analysis tools today are much, much better than that.  In fact it usually tends to be the other way around — 80%+ are real.  So in situations like that, you have to analyze the defects further.  After analyzing a few I can see many have been marked as “false positives” even though they are real.  When I challenge the customer it fits into either one of two categories:

1. I don’t care, that will never happen in my lifetime.

2. You’re wrong.

For #1, not everyone believes in defensive programming.  In some cases it doesn’t even make sense, in others I’ll still argue that it is important.  For example a customer once told me that the memory leak we identified is real but they don’t care, because it is initialization code.  Meaning when it starts, it leaks memory but it is closed off shortly after that.  I understand that sometimes you don’t have time to address every single issue, but what if someone decides to learn from you and copies that code.  Or later that code turns into something more than initialization.  It will happen.

For #2, typically the customer believes we are outright wrong.  Certainly in some cases we are, but in others we are just not providing the proper tools for the developers to understand the issue.  The best tool for that is what’s called “traceback”.  Traceback provides you clear instructions on the conditions and assignments of any detected defect.  One of my favorite examples is a buffer overflow.  The traceback below shows you the size of the array and where you are exceeding the bounds of that array.

In this example, it says we exceed that bounds by 15 (we are accessing the array from values 0-65).  The obvious question now is where on earth do you come up with 65?  When we expand the traceback, we can see the full details of that calculation.  The traceback proceeds through the details of every expression until we arrive at the right value, in this case 10+10+10+10+10+10+5=65.

False positives may be a way of life but when addressing them it is important to make sure you are not looking at a perceptual false positive.  The traceback tool is helpful in assessing not just perceptual false positives but other identified defects as well.  It will only make your life easier.

Wow, what a week last week. We’ve been busy spreading the word about Klocwork Insight 9.5 and all its on-the-fly-analysis goodness since late January and the reaction has been amazing, but last week took the cake.

First, Klocwork Insight 9.5 won its first award: the Military Embedded Systems Editor’s Choice. Here’s an excerpt from their write-up:

Software developers have enough on their plates without having to spend undue time analyzing their code. Sure, there is traditional source code analysis, whether done manually or by using techniques such as static or dynamic analysis, but Klocwork’s Insight 9.5 is intriguing because it strays away from the centralized batch-processing analysis modus operandi and instead implements on-the-fly, as-it’s-created code analysis.

Also last week, Klocwork Insight was one of 5 products to be awarded with CWE’s Official Certificates of CWE Compatibility.  This means our customers can rest assured that our products are “CWE Compatible”, based on CWE’s own criteria. Great news for all our customers who care about creating more secure code!

Klocwork Insight 9.5 was also a finalist for the Embedded World Award 2012, losing out in a close fought battle (so I’m told:) to Windriver and Willert for their software development tools. Congrats to the winners!

And that’s just in the the last few weeks since we launched the product.

Speaking of Embedded World, what a great event that was last week. Aside from the vast quantity of Nurnberg sausages we all ate, the sheer size of the event and quality of attendees was top-notch. The whole Klocwork team was run ragged all week hosting a customer event, doing demos at the booth, meeting partners, and just generally getting the word out.

So, good stuff all around for a product that was released barely one month ago!

Static analysis captures the current state of your code base and helps you answer key questions about the quality, security and maintainability of your software project.

Think Magic 8 Ball with build omniscience and powerful reporting tools. OK, maybe Magic 8 Ball isn’t a good analogy.

Answers to what questions, you ask? One we often hear from customers is: Where do I start?

A good place to start is a report that captures the distribution of defect types from your current build.  For example, we recommend that our customers glance over the Top 10 Issues report in our web-based build reporting tool, Klocwork Review while indulging in their morning cup of coffee:

Magic 8 Ball can't do this. Here's a defect distrubtion view of your build.

With this build snapshot and your caffeine jolt,  you can quickly identify defects of interest to your organization, such as null pointer dereferences and memory leaks. If you wish, you can set up filters (we call views) to show only these defect types in your report.

Your next step is to get your developers using static analysis on their desktops to prevent the injection of these high-priority defects into the build in the first place.

Once a policy of pre-checkin static analysis usage is put in place, pay attention to new defects injected into the build from that point on. If you see a spike in new defects, then investigate.

The magnitude of that y-axis is not what matters most; it’s the overall trend that counts.

For my next post, I’ll take a look at reports that track your cost of ownership and show you what success looks like.

In my previous post, It’s time to create a custom checker…, we looked at the considerations involved in deciding which checker to create–AST or path?

In this post, we’re going to use a custom checker to enforce an internal coding standard that extends the default set of checkers in our source code analysis tool.

To do this, I’ve called upon Steve Howard, our head of Partner Support in Europe, to get us started with an AST checker to accomplish our goal.

Steve has coached many customers through the checker creation process. In his experience, the appeal of custom checkers lies in their ability to enforce naming conventions and code constructions across organizations.

The standard we want to enforce is the use of a compound statement block rather than single statements as the body of a for loop. An AST checker is the way to go because detection depends solely on the syntax of the code itself and not runtime behavior.

See the example below:

To flag this violation, we need to instruct the checker to find all instances of for loop nodes that contain a Statement node as an immediate descendant.

A tool that shows you a visual representation of the AST for the test case is quite helpful in the checker creation process. Here at Klocwork, we use Checker Studio to:

• browse the AST structure of test cases,
• identify nodes of interest, and
• test XPath-like expressions that identify node types, qualifiers, conditions and variables to traverse the AST and flag the defect.

Note: If we wanted to enforce the compound statement rule in all loops, then we’d need to have one pattern (created using the XPath-like expression) for each possible kind, such as while loops and do while loops.

Armed with the test case, Checker Studio, and a syntax guide, Steve identified the following expression that flags the infraction:

// ForStmt [not (Stmt::CompoundStmt)]

Here’s how the test case and expression appear in Checker Studio:

Golden rules

Based on his experience, Steve has a number of golden rules that get you from idea to defect detection faster:

• Start simple: Use a simple test case that contains the defect you want to detect and work with one simple pattern at a time. Add more complexity as you go along
• Start rough and refine later: Don’t worry about false positives at first. In some cases it may even be easier to search for  instances that are OK and then negate the rule at the end
• Divide and conquer: With a more complex checker, work separately on each aspect of the defect you want to detect and then bring it all together at the end for testing in Checker Studio
• Watch your levels: Make the highlighting as relevant as possible for the issue you’re trying to find. For example, “// ClassType [MemberDecls[*]::MemberDecl]” will highlight classes that match, whereas “// ClassType/MemberDecls[*]::MemberDecl”  will highlight class members that match. The rule is the same, but the focus is different
• Weed out false negatives: Add negative examples (good code) to check for false negatives

For more information about our custom AST checkers, watch our Checker Studio video.

So it’s that time again, when our team finally gets to sleep for a day or two and get their lives back in order. What am I talking about?

Answer: Klocwork Insight 9.5 releases today and boy are we ever happy to see it go live — we’re “out there Jerry” and yes, of course we’re loving every minute of it.

This has been a long release, taking significant research and development to bring to fruition, all the while continuing to release more traditional shipments as we went, but finally culminating in a new, game-changing technology for source code analysis. On-the-fly, as-you-type, instant-like-for-reals, call it what you like… full-on, in-depth C/C++ analysis performed as the developer enters their code, using the “squiggly line” usability metaphor created by spell checkers. It’s one of those “why would you do it any other way” moments and we’re happy to be unique.

Not to be outdone, our web tools team has done amazing stuff with a complete redesign of our Review and Inspect tools, showcasing an awesome look and feel that leverages the toys that come with HTML5 and, amongst many other new capabilities, brings drag/drop pivot report design to the web for on-the-fly metrics and trending analysis that managers and development leads will just eat up.

To our customers, our partners and our friends in the industry, we’d like to say Welcome to Insight 9.5, hope you enjoy it.

You’ve been using source code analysis on your integration build or your desktop, or (ideally) both. And then there’s “a situation”.

The situation

Either you:

• Noticed a false negative you want detected, or
• Need a way to enforce a corporate coding standard, such as the requirement for the use of  a compound statement block rather than single statements as the body of a loop.

Now what?

Time to create a custom checker, that’s what. But what kind of checker?

Source code analysis involves two families of checkers, those that involve:

• Abstract Syntax Tree (AST) validation, and
• Code path analysis.

An AST provides a tree-based structural representation of the source code. An AST checker allows you to pinpoint problematic syntax using XPath or XPath-derived grammar to define the problem you’re looking for. AST checkers (our version is called Klockwork AST checkers, or KAST for short) don’t require program execution to run; they detect defects right away on source code.

Code path analysis, on the other hand, targets defects related to value tracking at program execution time. Instead of style violations, you’d use a path checker to answer questions such as:

• Is this newly-created object released before all aliases to it are removed from scope?
• Is this data object ever range-checked before being passed to an OS function?
• Is this string checked for special characters before being submitted as an SQL query?

To create a path checker, you don’t need to know how data is tracked by the checker. What you do need to know are the function types and values you want to track for the analysis starting point and the analysis end point where the defect (or event) is recognized and reported.

Which checker when?

Create an AST checker when the problem you want to detect:

• is a local defect
• does not involve program execution
• has to do with the way the program was written
• does not involve tracking a value
• does not involve a path

Create a path checker when the problem you want to detect:

• involves tracking a value
• has a starting point (where the analysis starts) and end point (where the defect is detected)
• involves program execution

Stay tuned for the next post in this series on best practices for AST checker creation.

For more information, see Writing custom checkers with Klocwork Extensibility or check out our member discussions in the C/C++ custom checkers forum.

–With files from CTO Gwyn Fisher

As a pretty avid Apple user (2 iPhones, 1 iPad2, iMac, iPod Touch, MacBook Pro, etc.), and the fact that I work in the business of software quality and security, I must admit that this article caught my attention. The article outlines how a well-known security researcher, who focuses on Apple, has found a software flaw in the iPhone and iPad, which could allow hackers to build malicious apps.What makes this even more scary is that the Apple Store may not catch these malicious apps.

To add another twist to this story, the researcher in question has been ejected from participating in Apple’s developer programs. Read about that here.

Are we now getting to the point where hackers are going to start trying more aggressively to exploit Apple products? In a survey done in 2010, over 50% of respondents thought Windows was either “very” or “extremely” vulnerable compared to only 20% for Apple. I wonder if that has changed? More importantly, do I need to start worrying about my daughter downloading the Archie comic app from the App Store?