Many of us have seen the # of lines of code (LOC) stats that get thrown around as a metric for illustrating how complex software development has become:

• The U.S. Army’s Future Combat System is estimated at 60 million lines of code (MLOC)
• The software that runs the Boeing 787 is almost 7 MLOC, triple that of the 777
• GM says future cars will have >100 MLOC (that sounds high, but hey, <insert GM joke here>)

So, yes there’s a lot of code out there, it’s growing, and it’s getting more complex. It’s tough to put these numbers into perspective… Jack Ganssle has a clever column that does just that:

• A million lines of code printed out would be 18,000 pages
• A million lines of code will typically have 100,000 bugs pre-test
• A million lines of code costs $20m to $40m

I won’t try to compete with Jack on the stats front since that column speaks for itself, and it’s not just the mission/safety critical systems that are trying to deal with the challenges of more code, more complexity, and less time (as in time-to-market). The consumer market is no different. Guess how much code is in the Xbox HD DVD Player? Not the whole system, just the player? 4.7 MLOC. Heck, you could fly an airplane with all that code (hopefully most of it can also be re-used on a non-obsolete disc format). Just as a point of comparison to today’s gaming systems, ask yourself how many software engineers worked on this “game system”:

Yes, I had to throw Digital Derby under the bus didn’t I? Not because I don’t have fond memories of playing this game that was about the size of carry-on luggage. In fact, I played it until the poor little conveyor belt just stopped working and I put up with the, ahem, limitations in the technology. I was actually quite patient (more patient than I am now with technology) and gave my little Digital Derby second and third chances to work properly.

More code, more complexity, impatient customers, and very little margin for error – these are the trends that are driving tool and process automation across the board in the SDLC.

Another year, another list of the most obvious things that competent developers should already know how to avoid? This one even has the NSA backing it, as well as the usual laundry list of pimping vendors attempting to make PR out of anything remotely related to homeland security… Quick, where do I sign up?

OK, perhaps that’s a bit cynical, but I have to say that my usual reaction to any web application-centric security laundry list is that most developers in that space write crap code, so why should we be surprised, or expect that a new list of world-ending doom and gloom will make any difference? I mean, why the heck is SQL Injection still a cause for concern? Haven’t we learned to write prepared statements yet? According to the widely quoted experts, SQL Injection and input validation alone comprised 1.5M exploits last year. Sigh…

Was nice to see some level-headed reporting in amongst all the hyperbole, however. My favorite quote of the day comes from the BBC article linked above, with Patrick Lincoln from SRI International saying:

“The real dedicated serial attacker will probably find a way in even if all these errors were removed. But a high school hacker with malicious intent – ankle-biters if you will – would be deterred from breaking in.”

Ankle-biters. Oh, the irony… All this money being spent on what? A barrier to deter script kiddies…

Maybe it’s just me, but I can’t help myself reflecting back to 1999. If you had code you’d written properly, that wasn’t lazy, that was expected to be around for a while, you simply weren’t cool enough to grab the big Y2K contracts. You weren’t cool enough to be worried about airplanes falling out of the sky. You were just a decent programmer doing a decent job.

Now of course, all the cool kids are worried about web site security. Why? Well as with everything that sounds noble and just on the outside, it probably has something to do with money, let’s face it. And cool-looking eye glasses with those little wire frames (oh yeah, you know you want them…). Because now you’re not a maintenance coder, you’re a code forensics expert. You’re not patching up some piece of garbage that should never have left the test server, you’re implementing cutting-edge perimeter security measures. You’re not hiding your head in shame about just how misguided our industry has become, you’re trumpeting it on the front page of every news outlet in the world…

And if you’re really cool you’re already fitting yourself out for your Steve Jobs lookalike outfit, because now, people, now we have the NSA telling us that the world is going to hell in a hand-basket, but if we’re quick enough, if we throw enough buzzwords around, maybe we’ll land ourselves a cushy gig as a maintenance programmer. Err.. I mean an architecturally-motivated, build-security-in, leading-from-the-trenches, application security infrastructuralizer kind of guy. Yeah, that sounds about right…

OK, don’t get me wrong… buzzwords aside, CWE is an awesome endeavor being run by some very smart and motivated people. And hey, this list will probably help me sell more of my software, which is always good, right?

But will this list of the Top 25 “dumb mistakes” change the world? Will it lead to a greater comfort for those of us waiting for the next terrorist attack to scare us senseless? Somehow I doubt it… But maybe, just maybe, it’ll stop grandma’s password making its way to the Ukraine next month. And that’s kind of the point. Web sites don’t run the world, but they can ruin it for you in a very personal way. Missiles aren’t going to fall out of the sky if your web site is vulnerable to path injection (or at least, let’s really hope not), but if your bank account is suddenly not yours anymore they might just as well have done.

But much as Hollywood’s brief dalliance with attempting to relate to developers by having some idiot repeatedly generate a 404 while “hacking the man” was obviously insane, hoping that a new list (“Larger and now with added scare factor!!!”) will have much impact on the security of things outside of web sites (like embedded device controller logic, or missile guidance systems, or air traffic management environments, or stop lights, or stuff that really, you know, matters) seems to this consumer to be equally off-base.

As an educational instrument I wish it all the success in the world. Better web applications would make lots of peoples’ lives easier. I’d really like to be able to visit any old web site and think that my personal details (“likes rock climbing, sushi and long beach walks…”) aren’t automatically open season for anybody with half a brain and too much time on their hands.

But here’s the bottom line: you don’t get security by enforcing a list. CWE is way, way bigger than this list of “Most Dangerous Programming Errors.” And it needs to be. So don’t go buying that shiny new product because it “Conforms to CWE 25” (or whatever this thing lands up being called). And don’t use the Top 25 to interview your new security consultant. Because what’s outside that list is just as important as what made the cut. Security is a big deal, it’s not a list.

David posted an interesting discussion on the usage of static analysis tools by developers to find security vulnerabilities.  As always the discussion with static analysis tools lean towards the false positive and false negative discussion.  But also David mentions their results are sometimes difficult to understand.  

This is one of the reasons Klocwork switched from a bytecode analysis tool for Java to a source code analysis tool.  As both have their advantages and disadvantages (and I admit I’m very biased here) we have certainly found that we have been able to reduce our false positive rates, find more issues (not to mention add new issues quickly) and provide more details on the results.

The reason for this is that the bytecode started to get noisy in Java 5 and even more so in Java 6.  Specifically one of the great advantages that static analysis tools have today is being able to show you the details of any issue it finds.  For example

public class Source {
     public static Source getInstance() { return null; }
     public int getValue() { return 32; }
     public Source() { getInstance().getValue(); }
}

This is a Null Pointer Exception issue. Although this is a very small example it is an inter-procedural issue and something that is missed by other defect detection tools. The important thing here is to help the developer understand this issue by tracing through the paths that make this happen.  In other words you need to know that getInstance() has the potential to return null and what paths bring us to the actual error.  This can get quite complex where it starts to not only span the same class but different methods in different classes.

Because of this extra noise this could be part of the reason your seeing new Java annotations being introduced.   Many of you may be familiar with the Java annotation that were introduced in Java 5.  This was part of a specification called JSR-250.  There is a new specification called JSR 305 for the purposes of allowing annotations to help assist tools that detect software defects.  Interestingly enough it is Findbugs leading the charge to add this in for Java 7.  Interesting to see if this specification happens.