In my previous post, It’s time to create a custom checker…, we looked at the considerations involved in deciding which checker to create–AST or path?

In this post, we’re going to use a custom checker to enforce an internal coding standard that extends the default set of checkers in our source code analysis tool.

To do this, I’ve called upon Steve Howard, our head of Partner Support in Europe, to get us started with an AST checker to accomplish our goal.

Steve has coached many customers through the checker creation process. In his experience, the appeal of custom checkers lies in their ability to enforce naming conventions and code constructions across organizations.

The standard we want to enforce is the use of a compound statement block rather than single statements as the body of a for loop. An AST checker is the way to go because detection depends solely on the syntax of the code itself and not runtime behavior.

See the example below:

To flag this violation, we need to instruct the checker to find all instances of for loop nodes that contain a Statement node as an immediate descendant.

A tool that shows you a visual representation of the AST for the test case is quite helpful in the checker creation process. Here at Klocwork, we use Checker Studio to:

• browse the AST structure of test cases,
• identify nodes of interest, and
• test XPath-like expressions that identify node types, qualifiers, conditions and variables to traverse the AST and flag the defect.

Note: If we wanted to enforce the compound statement rule in all loops, then we’d need to have one pattern (created using the XPath-like expression) for each possible kind, such as while loops and do while loops.

Armed with the test case, Checker Studio, and a syntax guide, Steve identified the following expression that flags the infraction:

// ForStmt [not (Stmt::CompoundStmt)]

Here’s how the test case and expression appear in Checker Studio:

Golden rules

Based on his experience, Steve has a number of golden rules that get you from idea to defect detection faster:

• Start simple: Use a simple test case that contains the defect you want to detect and work with one simple pattern at a time. Add more complexity as you go along
• Start rough and refine later: Don’t worry about false positives at first. In some cases it may even be easier to search for  instances that are OK and then negate the rule at the end
• Divide and conquer: With a more complex checker, work separately on each aspect of the defect you want to detect and then bring it all together at the end for testing in Checker Studio
• Watch your levels: Make the highlighting as relevant as possible for the issue you’re trying to find. For example, “// ClassType [MemberDecls[*]::MemberDecl]” will highlight classes that match, whereas “// ClassType/MemberDecls[*]::MemberDecl”  will highlight class members that match. The rule is the same, but the focus is different
• Weed out false negatives: Add negative examples (good code) to check for false negatives

For more information about our custom AST checkers, watch our Checker Studio video.

My first 2 posts looked at 2 different eras of Static Code Analysis, the Early Years and the Early 21^st Century. The SCA solutions of these times were revolutionary, and helped software development teams a great deal. But they had their warts.

In the final post in this series, I’m going to introduce you to the present day Static Code Analysis technology and how it is impacting developers.

The Present Day

I’m a huge fan of Reece’s Peanut Butter Cups. I love them. I keep active so I don’t feel guilty eating them. In a strange, convoluted way, the 3^rd generation of static code analysis tools are like this delicious combination of chocolate and peanut butter. Let me explain.

Eureka! Now developers are able to perform an analysis locally, using their development environment of choice, while still getting the high accuracy and consistency that was previously only possible by checking in their code and waiting for the integration build to take place.

Think about the ramifications of this:

• cleaner code is being checked in
• the ‘rinse-repeat’ vicious cycle of rework is drastically reduced
• quality teams are now able to focus on testing the product’s functionality rather than spending cycles uncovering something that could easily and quickly be found by automated tools.

Mmmm-mmmm good. Sounds like a win-win-win to me!

I think the best thing about these 3rd generation tools is simply the fact that developers are now able to resume ownership of the quality and security of the code they are producing.

Well, I hope you enjoyed this walk down memory lane. I sure did. Now I’m looking for spare change because I see a trip to the vending machine in my immediate future.

If you want to know more about the 3rd Generation tools, feel free to drop me a line.

Creating a source code analysis (SCA) engine is a balancing act, a decision process of where you believe the most value can be found along the spectrum that is the signal-to-noise ratio of the detection process. At one end lies the realm of massive noise and hopefully complete coverage, whilst at the other is the quiet calm of the theoretically useful but ultimately useless realm of no noise, but ultimately no signal either.

That may sound counter-intuitive. Shouldn’t a zero noise point on the spectrum be accompanied by an infinitely strong signal? Perhaps in the world of DSP this is true, but in the world of SCA reducing noise comes right along with a reduction in detection capability – it’s unfortunately almost a straight-line correlation.

So if we assume that we’re trying to balance a couple of dials on our theoretical tuner, we might start by reducing or dampening noise – it’s the most obvious place to start, after all. Nobody likes to listen to their favorite FM station through the curtain of hissing and popping that accompanies the act of driving through a major city.  Likewise no developer likes sifting through a long list of bogus detection errors in order to find the hidden gems. But to drag out the analogy, assume that the only way of reducing hiss on your FM signal is to turn down the volume… now you’ve got less hiss, but also less Bruce Springsteen goodness to accompany it.

Balance is what we need here, obviously. Enough Boss to make us ignore the hiss, or to put it in a more SCA-like context, enough interesting bugs to make us ignore the incorrect, or the irrelevant (correct detections on the part of the engine that the developer just doesn’t care about, e.g. low memory conditions in a memory-insensitive environment).

Consider the following simple example that clearly lies “on the line”:

    void foo(char* s, int a)
    {
        char* s1 = s;
        if( a > 0 )
            *s1 = 'a';   // potentially use an uninitialized ‘s1’
    }

    void bar(int m)
    {
        char *s;
        foo(s, m);       // s is not initialized prior to calling ‘foo’
    }

So… to report, or not to report?

Lacking any other information, it is obvious that function ‘foo’ interacts under certain situations (when parameter ‘a’ is positive) with parameter ‘s’ (aliased as local variable ‘s1’). As we have no knowledge about the provenance of parameter ‘s’ when analyzing ‘foo’, however, there’s nothing here to cause a report and so we squirrel away the knowledge of what ‘foo’ does for later use.

When analyzing ‘bar’ we know what ‘foo’ does, and we know we’ve got an uninitialized local pointer, ‘s’. But again we’re lacking enough knowledge to know the valid values, or ranges, that parameter ‘m’ may take. There are definitely a set of circumstances here in which we know a problem will occur (if parameter ‘m’ is positive), and a set of circumstances in which we know a problem will not occur (if parameter ‘m’ is zero or negative) – this much is encoded in the functional behavior of ‘foo’. But is it a defect, or should we filter out the report in favor of providing only those situations in which we can be “sure” the bug not only exists, but can be proven to be exercised?

There’s the art of balance in a nut-shell, and it revolves around the phrase “lacking any other information.” In the ideal world, lacking any restrictions in terms of time, memory or computing power (or indeed actual from-the-wall power, as we have to worry about now), we might defer all such decisions until we categorically know that a particular data value is passed down the call graph far enough to get to ‘foo’. But in the real world of multi-million LOC projects, that approach simply can’t scale.

And so, calling on balance as our friend, we can bias a localized decision to report or not, given that we know to at least one order of approximation that bad things could happen here. Different engines pronounce that bias differently, leading to one of the greatest divides between prevalent solutions.

Now ask yourself, as the developer, is it a worthy report if you know that 10 levels up the call graph there’s a check on what eventually becomes parameter ‘m’ to ensure that it’s never positive? Perhaps you’d automatically classify this as a false positive and, annoyed at the tool, move onto the next report. Or perhaps, seeing the size of the gap in the call graph, you might just choose to code defensively, initializing ‘s’ to NULL in ‘bar’ and adding guard code to ‘foo’ because, hey, you never know.

And as we’ve all seen so many times over the years, “you never know” might just as well be written “and so it came to pass…”

In my last post, I took us back in time to an era of bad fashion, questionable music, legendary television shows, and source code analysis tools that were made specifically for software developers. It was the 1970s. In this post, I fast forward to just after the turn of the century to discuss the next evolution of static analysis tools.

The Early 21^st Century

Not long after we first viewed hairy-footed Hobbits on the silver screen, and the sham that was affectionately known as Y2K, a new generation of source code analysis tools emerged to cure the errors of the first-generation tools.

These new tools looked beyond the syntactical analysis of previous tools, and instead provided inter-procedural and data-flow analysis. Low hanging fruit was definitely not the target for these tools.

These new techniques were serious–finding complex defects that could impact code quality and security, and they did that while ensuring that the “noise” (i.e. false positive rate) was greatly reduced compared to the first-generation tools. In addition to local defects, they were now identifying resource management issues, security vulnerabilities, concurrency issues, and so on. These were serious defects that,  if left undetected and unfixed, had the potential for massive problems to the code stream.

In order to perform this much deeper analysis, a fundamental change in the analysis techniques had to occur. These engines needed an unfiltered view of the entire code stream, and so they became tightly integrated with the integration build process.

Umm, Houston, we have a problem. If the analysis takes place at integration build time, then that means the analysis is no longer being initiated by the developers. Source code analysis tools became centralized and moved into a more downstream process such as part of a code audit function.

Developers were now being told they created bugs well after they actually checked in the code. They had already moved onto something entirely different, so now bringing them these day-old, or week-old defects was certainly not the most productive use of their time. It is well documented that the earlier you find defects in your code, the more cost effective it is to fix them, so you can clearly see the problems with these second-generation tools.

If only there was a way to bring these second-generation analysis capabilities to the developer desktop. More about that in my next entry.

Our dilemma: How do we remove the barriers to knowledge about Klocwork’s toolset and developer best practices for creating high-quality code?

The answer: Klocwork Developer Network–a new online portal designed for learning, sharing and discussing all things source code analysis. We have had a lot of fun and a few sleepless nights as we assembled industry knowledge, online forums, computer-based training, best practices from industry experts, and lots of reference and learning resources.

A significant portion of the content on the Developer Network is open for public consumption. By registering and logging in, you get additional videos, demos, CBT and more.

We have a lot of fresh content to add to the site in the upcoming weeks and months, and we want to hear from you about what you would like to see. Why not register now at developer.klocwork.com? Then tell other Klocwork users about the portal too.

Visit Klocwork’s Developer Network at developer.klocwork.com.

Already a my.klocwork.com user? Access the Klocwork Developer Network using your existing my.klocwork.com login. (But note that my.klocwork.com remains the place to go for support tickets and for FTP access to the latest software releases.)

As a product manager, the only backlog I typically care about is my product backlog. Do I have the right stories in there? Do the stories have enough detail? Are they properly prioritized? You know, that kind of stuff. Today, however, I’m going to write about a very different backlog, that is the static analysis defect backlog.

A static analysis backlog is created when you run a static analysis product on your code base for the very first time. Chances are pretty good that the first analysis is going to list a large number of defects, some that are without question real, and some that perhaps are not. Do not freak out! This is the first time that analysis engine has ‘laid eyes’ upon your code and it is going to flex its muscles and show you any weaknesses it believes exist. So how does one deal with this? Here are a few strategies to help you:

1) Don’t boil the ocean. Before you even run that first analysis, don’t have a “wouldn’t it be cool” moment, where you decide to turn on every single rule the analysis engine has. There is a reason why static analysis tools haven’t turned on everything.  They are showing the most accurate and critical issues first.  So unless you have unlimited time and resources, your best bet is to start with a core set of rules and run the analysis based on that set. This core set of rules should include things such as memory/resource leaks, buffer overruns, null pointer dereferences, uninitialized variables, and so on. Add other rules once you have this core set under control.

Is your issue backlog making you cross eyed? Try these coping strategies.

2) Baseline your defects. Consider that first analysis your baseline and choose to ‘park’ them for the time being. Chances are the product that the analysis was run on is one that has already been released to the public, and in good working order. Zero out these defects for now, and start to triage them, which leads into strategy #3.

3) This is going to sound pretty obvious, but when it comes to managing your issue backlog start looking at the most critical issues first. These are the ones that are most likely to cause a failure of some sort, so determine if these issues are real, and if so, fix them immediately. Once you’re done with the most critical issues, move to the next level of severity, and continue on that way.

4) Finally, tune your analysis. Any good vendor will allow you to tune your analysis. The benefits of tuning are twofold; 1) you can find code issues that would otherwise go undetected and, 2) reduce the number of issues that the engine reports incorrectly in the context of your source code. You should think of ways to give the tool more context about your code base to increase accuracy.

If you follow these suggestions, you’ll definitely have a better grasp of your bug backlog, and you’ll be able to execute on reducing that backlog quickly and efficiently. If you don’t, then at some point, you may feel a little like the critter pictured here.

If there are any other strategies you’ve tried to deal with your bug backlog, leave a comment or two. I’d love to hear about them.

Part of my life has been spent in the manufacturing sector working with industrial automation devices, but the discovery of the Stuxnet virus is the first time I’ve ever heard of specifically virus targeting and even rootkitting a PLC (programmable logic controller) or  SCADA (supervisory control and data acquisition) network.

When working in industrial plants, we took the standard precautions with regard to Windows viruses and even started to add virus protection for Linux, but never did it occur to any of us that the industrial automation equipment might be at risk. Whenever the subject was even brought up, which was rare in itself, there were the standard arguments:

• Oh, it’s on a physically separate network (or VLAN configuration), only USB (thumb/flash) drives are allowed and they’re virus checked before use.
• Oh, it’s running a completely different processor/operating system/architecture – there’s no way it can be infected.

The consequences of infection are severe.   These devices run everything from our nuclear power plants to complex manufacturing assembly lines, aircraft controls (FADECs) and chemical refineries, just to name a few.  In its most basic of functions, industrial automation is used for two purposes: to keep humans safe and to produce products for less cost.  Interrupting either of these is going to kill someone or cost a company a large chunk of change.

So, what does this all mean?  It means that industrial automation and PLC vendors had better start hardening their solutions for security vulnerabilities and elevate the quality of their firmware and software components using security vulnerability tools such as Klocwork’s static analysis just as the general computing industry has done for the past 30 years.

For an in-depth analysis and timeline, refer to either Symantec’s whitepaper on their Stuxnet analysis or the work done by ESET on their version of Stuxnet.

Most code reviews are done in-person, 60%  according to data  from a Forrester Consulting study commissioned by Klocwork.  So how do you accommodate remote sites, out-of-office employees  or off-shore development shops?

Most software developer teams will face some form of remote development challenge during their careers or product cycles.  As demonstrated from the data above, the breakdown of remote need is as follows:

• 76% use some form of outsourcing,
• 64% have some developers  located outside of the main campus,
• 40% of reviews are conducted with remote participants.

You can’t let development come to a grinding halt simply because a critical team member is not physically available at the scheduled time or location.  For most organizations, code reviews need to be performed and employee travel is not the solution for cost and timing reasons.  This has driven the adoption of lightweight review processes and new tools that support it.

Klocwork built a code review tool for this express purpose.  Other ones exist like Code Collaborator and the open source Review Board .  How do you support your remote code reviews?  Email?  Wiki? Or a purpose-built tool like one of the ones mentioned?

I love this quote by Carl Ek from  Code Integrity solutions:

There are 0010 0000 kinds of people in the world: Those that understand the difference between Big Endian and Little Endian, and those that do not.

Issues with Endianism and processor architecture ports are becoming more and more common these days as more desktop source code moves into different arenas.  Gone are the days when the 32-bit memory model or little-endian format dominate. Software changes are required to support the growth occurring not at the desktop, but in the server  and mobile platforms.

Mobile devices especially have opened a Pandora’s box of Endian and memory problems, with variety of processor architectures with ARM^[1] leading the way.  Add to this mix,  end-consumers are demanding desktop features like Adobe Flash or Office apps on mobile devices, many a stable codebase will fall apart when ported to either mobile or server.

For developers porting to different platforms, there are some significant challenges.  Just to list a few:

• CPU optimizations need to be reviewed
• inline assembly calls require rewriting or removal
• machine word (WORD) allocations may require refactoring
• any binary data exchanged over the network stacks require verification

None of these are  new, they’re just not a common skillset for most developers.

Source code analysis can be a boon in two ways. Firstly, in the planning phase by helping you determine the breadth of the effort,  and secondly by identifying any existing  issues, particularly of the memory allocation and Endian varieties.

For more in depth information, there are two recent articles available from Dr. Dobbs:

• Porting to 64-bit Platforms by Irving Rabin
• Detecting Endian Issues with Static Analysis Tools by Carl Ek

^[1] Note: Some ARM processors support both big and little Endian formats.

Mark Grice is a pretty unflappable guy, but when you ask him a question about barriers to successful adoption of Source Code Analysis (SCA) technology, he starts to splutter.