Static analysis captures the current state of your code base and helps you answer key questions about the quality, security and maintainability of your software project.

Think Magic 8 Ball with build omniscience and powerful reporting tools. OK, maybe Magic 8 Ball isn’t a good analogy.

Answers to what questions, you ask? One we often hear from customers is: Where do I start?

A good place to start is a report that captures the distribution of defect types from your current build.  For example, we recommend that our customers glance over the Top 10 Issues report in our web-based build reporting tool, Klocwork Review while indulging in their morning cup of coffee:

Magic 8 Ball can't do this. Here's a defect distrubtion view of your build.

With this build snapshot and your caffeine jolt,  you can quickly identify defects of interest to your organization, such as null pointer dereferences and memory leaks. If you wish, you can set up filters (we call views) to show only these defect types in your report.

Your next step is to get your developers using static analysis on their desktops to prevent the injection of these high-priority defects into the build in the first place.

Once a policy of pre-checkin static analysis usage is put in place, pay attention to new defects injected into the build from that point on. If you see a spike in new defects, then investigate.

The magnitude of that y-axis is not what matters most; it’s the overall trend that counts.

For my next post, I’ll take a look at reports that track your cost of ownership and show you what success looks like.

Part of my life has been spent in the manufacturing sector working with industrial automation devices, but the discovery of the Stuxnet virus is the first time I’ve ever heard of specifically virus targeting and even rootkitting a PLC (programmable logic controller) or  SCADA (supervisory control and data acquisition) network.

When working in industrial plants, we took the standard precautions with regard to Windows viruses and even started to add virus protection for Linux, but never did it occur to any of us that the industrial automation equipment might be at risk. Whenever the subject was even brought up, which was rare in itself, there were the standard arguments:

• Oh, it’s on a physically separate network (or VLAN configuration), only USB (thumb/flash) drives are allowed and they’re virus checked before use.
• Oh, it’s running a completely different processor/operating system/architecture – there’s no way it can be infected.

The consequences of infection are severe.   These devices run everything from our nuclear power plants to complex manufacturing assembly lines, aircraft controls (FADECs) and chemical refineries, just to name a few.  In its most basic of functions, industrial automation is used for two purposes: to keep humans safe and to produce products for less cost.  Interrupting either of these is going to kill someone or cost a company a large chunk of change.

So, what does this all mean?  It means that industrial automation and PLC vendors had better start hardening their solutions for security vulnerabilities and elevate the quality of their firmware and software components using security vulnerability tools such as Klocwork’s static analysis just as the general computing industry has done for the past 30 years.

For an in-depth analysis and timeline, refer to either Symantec’s whitepaper on their Stuxnet analysis or the work done by ESET on their version of Stuxnet.

As new words and concepts diffuse in to wider use, their definitions become simpler or broaden to cover more scope.  Like the kid’s telephone game, each time the concept is passed to another developer, the information gets a little more muddled. In software development, declaration, macros, syntax and other programming constructs have to be exact or the compilers will fail.  Yet, when developers discuss concepts about programming, most of the time, that precision of language is lost.

The telephone game  seems to have happened to refactoring.  I subscribe to what would be consider the  ”classic” definition of  refactoring: the process of optimizing or extending a class  but leaving the existing exposed interface alone.  It seems that refactoring has been generalized from that definition into covering all activities related to touching old code.  Even worse, it has become an excuse to rework someone else’s code and then bitch about how bad it was.  Just look at the twitter stream for refactoring for a never-ending torrent of  abuse.

There is one simple way to tell if your efforts are truly a refactoring or not.  Did you break any of the unit tests?  If you did, then you are rewriting code instead of refactoring it and you had better update all the test cases while you’re at it.  Don’t get me wrong, there are times to rewrite software, but they are few and far between and, in my experience, it almost never pays off.

Refactoring isn’t just a nice philosophical idea; it supports one the most basic concepts in software development–backwards compatibility.  If you want to keep your customers and enjoy that paycheck, don’t break features or APIs in products without good reason or  get buy-in before release day.

To refactor the medical phrase “First, do no harm“, for developers it should be “First, break no test”.

I’ve talked about different metrics that are used to measure quality and the metrics that developers would use in practice.  But what about the tools themselves?  How are you measuring the value you are getting out of these tools?

In terms of static analysis, one obvious measurement is simply the bug fixes you have made.  Most organizations have a number they use to define the cost savings for each bug.  Using some research data from IBM puts the cost of fixing a bug before a release at 40-50 times cheaper.   Fixing a bug after release is the extreme and hopefully you’re finding these issues earlier and that’s where static analysis comes in.

I would think any static analysis tool offering would give you that simple graph.  Below is a couple of examples where you can track the fixes per component or per owner.

Bug fixes per compoent

Bug fixes per owner

One other thing that I think is very important is where you’re saving money.  When you run static analysis for example you have a couple of choices where you can find defects.  After the integration build or right at the developers desktop before they check-in their code.  Obviously if the developers are finding and fixing their issues while they code and before they even get checked in then it is saving even more money for the company.  Good static analysis tools will help you see the value of using it at the desktop.  A simple counter is all you need to see how many bugs were fixed in the code prior to check-in.  Immediate ROI from using static analysis.

As readers know, we’ve been talking about code reviews pretty regularly here and elsewhere over the past few months. To continue that discussion, here’s a question we run into often: are in-person code reviews as the primary way to communicate, by definition a bad thing?

Here’s some more data from the Forrester Consulting study commissioned by Klocwork that shows the majority of respondents still conduct in-person reviews… elsewhere in the survey only 36% of respondents indicated that they worked on a centralized team with everyone in one location. So that means, if 60% still conduct in-person reviews, they’re likely excluding valuable contributors to the review.

Data that shows majority still conduct in-person code reviews

Is this practice just being done because “that’s the way it is” or are there good reasons for in-person meetings being the primary way to review code? I could see the odd in-person meeting being necessary for a variety of reasons but given how distributed teams are these days and the variety of tools available to effectively review code remotely, it doesn’t seem that efficient.

There’s a general philosophy gaining more prominence around meeting reduction, whether in software development or elsewhere. We’re seeing many organizations question why their code review process needs to be in-person when it excludes people who aren’t co-located and generally takes up too much of people’s time. What are you seeing?

Mark Grice is a pretty unflappable guy, but when you ask him a question about barriers to successful adoption of Source Code Analysis (SCA) technology, he starts to splutter.

Just came across this post about the 5 code metrics you need to watch.  I thought it was worth mentioning as I just blogged about this below (including something similar a while back).  These are interesting metrics and more high level, but certainly important.  I like labeling duplicated code as something important.  I think we often forget how much we reuse code and have the same mistakes in many places.

A recent article on the top five causes of poor software quality and top 5 non-technical mistakes inspired me to also provide a top five on software quality bugs.  Here is my top 5 list of bugs (with some simple examples) that I see time and time again looking at customer code:

1.    Null Pointer dereference

This is far and beyond the most common issue that I see time and time again.

void npd_gen_must() {
int *p = 0; // NULL is assigned.
*p = 1;  // pointer is dereferenced
}

Now this example is pretty basic and if you ever did something this obvious, maybe it was time to re-evaluate your development skills.  The idea is simple, you assign NULL somewhere then dereference it at some point later.  This is usually missed under a complicated control flow (many conditionals).  Or even more common is the fact that I see memory is allocated, but is never checked against NULL.  Now, some organizations don’t care about this but I would hope anyone doing embedded development is all over it.

2.    Null pointer dereference from function

This is really the same thing but with one very important difference.  This deals with issues from functions.

void xstrcpy(char *dst, char *src) {
if (!src) return;
 dst[0] = src[0];
}

char global;

char *xmalloc() {
  if (global) return &global;
  return 0;
}

void npd_func_might(int flag, char *arg) {
  char *p = &arg;
  if (flag) p = xmalloc(); // xmalloc() may return NULL
  if (arg) { p = arg; } // p may get a new value here
  xstrcpy(p, "Hello"); // p will be dereferenced in xstrcpy()
}

It is this inter-procedural (spanning multiple files/functions) context that is often overlooked.

3.    Memory leaks

I have yet to find a programmer in the C/C++ world who doesn’t know this intimately.  Sadly they happen, a lot.

void foobar(int i) {
  char* p = new char[10];
  if(i) {
    p = 0;
  }
  delete[] p;
}

Here we have dynamic memory stored in ‘p’ and allocated through the function ‘new[]‘ at line 3 and is ultimately lost at line 5.

4.    Array index out of bounds

Again, most people know what these are but there are so many variations of this that they are always inevitable.

int main() {
  char fixed_buf[10];
  sprintf(fixed_buf,"Very long format string\n");
  return 0;
}

The string is 24 characters so at line 4 the array index of ‘fixed_buf’ may be out of bounds.

5.    Uninitialized variables

int foo(int t) {
  int x;
  if (t > 16) {
    x = 1;
  } else if (t > 8) {
    x = 2;
  }
  return x + 1;
}

The value of variable ‘x’ can be used at line 8, when it might be uninitialized.  I always found these surprising that these come up as they are pretty basic.  But I tend to only see these in complex control flow paths.  So the developer might check for these under normal conditions but forgot on some path.  Especially for legacy code this might not bite you until you change something later on.

So that’s it.  These examples are pretty simple and certainly not reflective of the real world (or at least I hope not).  Later I will post the same idea for Java code.

Another year, another list of the most obvious things that competent developers should already know how to avoid? This one even has the NSA backing it, as well as the usual laundry list of pimping vendors attempting to make PR out of anything remotely related to homeland security… Quick, where do I sign up?

OK, perhaps that’s a bit cynical, but I have to say that my usual reaction to any web application-centric security laundry list is that most developers in that space write crap code, so why should we be surprised, or expect that a new list of world-ending doom and gloom will make any difference? I mean, why the heck is SQL Injection still a cause for concern? Haven’t we learned to write prepared statements yet? According to the widely quoted experts, SQL Injection and input validation alone comprised 1.5M exploits last year. Sigh…

Was nice to see some level-headed reporting in amongst all the hyperbole, however. My favorite quote of the day comes from the BBC article linked above, with Patrick Lincoln from SRI International saying:

“The real dedicated serial attacker will probably find a way in even if all these errors were removed. But a high school hacker with malicious intent – ankle-biters if you will – would be deterred from breaking in.”

Ankle-biters. Oh, the irony… All this money being spent on what? A barrier to deter script kiddies…

Maybe it’s just me, but I can’t help myself reflecting back to 1999. If you had code you’d written properly, that wasn’t lazy, that was expected to be around for a while, you simply weren’t cool enough to grab the big Y2K contracts. You weren’t cool enough to be worried about airplanes falling out of the sky. You were just a decent programmer doing a decent job.

Now of course, all the cool kids are worried about web site security. Why? Well as with everything that sounds noble and just on the outside, it probably has something to do with money, let’s face it. And cool-looking eye glasses with those little wire frames (oh yeah, you know you want them…). Because now you’re not a maintenance coder, you’re a code forensics expert. You’re not patching up some piece of garbage that should never have left the test server, you’re implementing cutting-edge perimeter security measures. You’re not hiding your head in shame about just how misguided our industry has become, you’re trumpeting it on the front page of every news outlet in the world…

And if you’re really cool you’re already fitting yourself out for your Steve Jobs lookalike outfit, because now, people, now we have the NSA telling us that the world is going to hell in a hand-basket, but if we’re quick enough, if we throw enough buzzwords around, maybe we’ll land ourselves a cushy gig as a maintenance programmer. Err.. I mean an architecturally-motivated, build-security-in, leading-from-the-trenches, application security infrastructuralizer kind of guy. Yeah, that sounds about right…

OK, don’t get me wrong… buzzwords aside, CWE is an awesome endeavor being run by some very smart and motivated people. And hey, this list will probably help me sell more of my software, which is always good, right?

But will this list of the Top 25 “dumb mistakes” change the world? Will it lead to a greater comfort for those of us waiting for the next terrorist attack to scare us senseless? Somehow I doubt it… But maybe, just maybe, it’ll stop grandma’s password making its way to the Ukraine next month. And that’s kind of the point. Web sites don’t run the world, but they can ruin it for you in a very personal way. Missiles aren’t going to fall out of the sky if your web site is vulnerable to path injection (or at least, let’s really hope not), but if your bank account is suddenly not yours anymore they might just as well have done.

But much as Hollywood’s brief dalliance with attempting to relate to developers by having some idiot repeatedly generate a 404 while “hacking the man” was obviously insane, hoping that a new list (“Larger and now with added scare factor!!!”) will have much impact on the security of things outside of web sites (like embedded device controller logic, or missile guidance systems, or air traffic management environments, or stop lights, or stuff that really, you know, matters) seems to this consumer to be equally off-base.

As an educational instrument I wish it all the success in the world. Better web applications would make lots of peoples’ lives easier. I’d really like to be able to visit any old web site and think that my personal details (“likes rock climbing, sushi and long beach walks…”) aren’t automatically open season for anybody with half a brain and too much time on their hands.

But here’s the bottom line: you don’t get security by enforcing a list. CWE is way, way bigger than this list of “Most Dangerous Programming Errors.” And it needs to be. So don’t go buying that shiny new product because it “Conforms to CWE 25” (or whatever this thing lands up being called). And don’t use the Top 25 to interview your new security consultant. Because what’s outside that list is just as important as what made the cut. Security is a big deal, it’s not a list.