Mark Grice is a pretty unflappable guy, but when you ask him a question about barriers to successful adoption of Source Code Analysis (SCA) technology, he starts to splutter.

A while back, I talked about how I keep running into organizations that seem to go out of their way to make developers’ lives hell.  I’ve run into several examples where developers had to switch between different environments just to write and compile code.  That’s as productive as watching paint dry and as much fun as rearranging the deck chairs on the Titanic.

For teams that want to run source code analysis in these types of environments (or any kind of dev tooling, frankly)  it’s very difficult for vendors to support.  I did my usual PM grumbling about these environments but since that post exactly 1 year ago I’ve come to realize that these environments are a reality and we need to figure out a way to support them.  Maybe it’s not productive but organizations are making it work.  I’m sure they would even argue that they have made it productive (good luck to them).  It’s for this reason that Klocwork has given in and instead of pointing our finger and making fun (I swear I never did), we’ve decided that it’s in our best interest to make sure we provide these customers with the capability to run static analysis.

A couple of releases ago, Klocwork introduced a new tool called Klocwork Desktop that provides Klocwork command line users with the same graphical capabilities that one would get from Visual Studio or Eclipse.  This tool was great for users who never used an IDE.  With Klocwork’s 9.1 release we have extended Klocwork Desktop’s reach by providing a remote capability that’s designed to support the type of environments described above.  Using Klocwork Desktop in remote mode allows users to view their source and detected-issue information when Klocwork Desktop does not have direct access to source files or defects, yet still get the benefits of finding and fixing your defects before you check-in your code.

One really cool feature that is part of this is the “you’ve got mail” notification.  At first, I have to admit this is something that worried me.  If I had to label one thing as a productivity drain it was those annoying alerts you get of new email coming in.  Of course right in the middle of doing something important you get distracted by a new email with plans for the next party (or in my case hearing about the kids latest poop explosion).  The first thing I always do is turn it off.  But in the case of finding bugs while coding, it only makes sense to give you these notifications in a heartbeat.  So you can actually be writing code on some machine in Jakarta and automatically your machine in San Jose is alerting you of bugs.  Pretty neat stuff.

Just returned from my second stint on the Agile in Action roadshow with our friends from Electric Cloud, Perforce, and VersionOne, this time visiting the cities of Toronto, Philadelphia and Chicago. Rather than going into minute detail (and the fact it is a Friday afternoon before a long weekend), I thought I would share a few random observations from this trip:

• Organizations (and individuals) are begging for as much information and guidance as they can get on Agile and tools for Agile, and are willing to give up a days in the office and brave horrific traffic to get it
• Teams that are 6 to 9 months practicing Agile think they’re novices, but in reality are seasoned veterans and have lived through most of the nightmares newer teams are currently facing
• Toronto cab drivers have a random-number generator for their “flat-rate” fares from the airport
• The majority of our audience would rank low to medium on both their knowledge and their adoption of Agile…they all want to go Agile, they just don’t know where to start (or if they were started, how they could improve things)
• Window seats suck, but not as much as middle seats
• Developers do code reviews, but don’t like doing them…
• …but you could always count of the one guy in the audience who claimed to like them…obviously someone’s living in denial
• And finally, if you are in 3 different hotels in 3 nights, keep the sleeve your room key comes in on you at all times…I guarantee you’ll forget your room number at least once during the trip.

In a previous post I discussed the process where we practice dogfooding.  This is the process of using Klocwork on Klocwork (KonK).  We started this program several years back with the hopes that we would learn some valuable lessons about usability, performance and anything else that would give us an edge.  The truth is that KonK has consistently allowed us to test our design assumptions early by allowing our own developers to use Klocwork as part of their development.

One of the unexpected results was inadvertently uncovering data that further validated for us the importance of bugs found by a static analysis tool. We correlated testing-found issues and KonK (static analysis) issues assigned to each developer. The result as this graph shows (note: graph is using a logarithmic scale and is a few years old), is there was a very high relationship between the two. Those familiar with bugs found by static analysis tools know that they are often quite different in nature from a bug found by testing where you’re testing requirements, yet developers with a high count of testing-found issues also had a high KonK count. In some cases they pointed to the same problem, in others an indication of the overall bad to the bone problems in some of the new components.

Since we already eat our own dogfood, we’re already believers in the use of static analysis, but this kind of data is a great example of the benefit of finding issues early using static analysis

I just got back from a visit to the Valley and had an awesome week in San Jose/San Fran.  I even had time to play a bit of the tourist this time (I ran the Golden Gate bridge/Presidio).  All that was fun, but what I always enjoy is the conversations we had with customers and prospects at this year’s ESC SJ 2010 conference.

It is always interesting listening to their successes and teasing out the trending topics and new issues that matter to development teams.  Here are the top three themes that caught my ear this year:

1. The economic rebound is well underway, with growth and optimism from every quarter.  It may be too early to see results on the balance sheets, but the positive attitude is back.
2. Embedded developers are searching for enterprise-class developer productivity tools, like static analysis, for even tiny codebases (less than 40 kLOC).
3. By far, the most-often raised topic in one-on-one conversations was coding standards, with MISRA C and C++ as the favorite.  MISRA’s time has definitely arrived for the embedded community.

So all in all, a great time, and looking forward to next year.

Just got back from our 5^th annual Klocwork Customer Advisory Board, graciously hosted in hot and sunny Phoenix, Arizona by one of our top customers. These events are now running like, umm, clockwork, as we have come up with a winning recipe that mixes a nice combination of Klocwork delivered material and customer delivered material over the course of 2 days. We had a great mix of ‘seasoned veterans’ and new members to this year’s CAB which worked out extremely well. We decided to add a keynote speaker to kick our meetings off, which by all accounts was very well received. I think that will now be a permanent part of our meetings going forward.

While we were inside a meeting room for the better part of 2 days, we did have some time to experience the great outdoors in Phoenix.  The afterhours ‘event’ this year was riding around the Tonto National Forest…at night, without headlights…in Hummers. Not your garden-variety Hummers either, but bad-ass Hummers that seem unstoppable. Throw in some night-vision goggles, scorpions, cacti, and tarantulas and you’ve got yourself one heck of an outing. If you’re ever in the Phoenix area, you have to check out Desert Storm Hummer Tours.

Every year I come back from CAB wondering how we will top the last one, and it is no different this year. This CAB has set the bar very high, but I’m sure we will find a way to exceed it.

Our partners at Code Integrity have a good blog that touches on many of the benefits and barriers to static analysis within a development organization. They have an interesting post on “False false positives” – a great phrase that captures one of the key challenges in developer adoption of the technology.

While increased sophistication means that static analysis tools can catch more problems with a higher degree of accuracy, the burden increases on the reviewer of the results to interpret them correctly. If you were grep’ing through some code for something you can quickly review (and dismiss) many of the results because you understand what your “analysis” is doing. With static source code analysis, this is much less apparent.

We see many engineers look at a complex bug report and not take the necessary time to understand the problem and fix it. This is mostly because they don’t understand what the static analysis tool is doing and how deep it is analyzing the code. The result is a real bug being marked as a false positive – or a “false false positive” if you will. These bugs then disappear off the queue never to be seen again – a lost opportunity.

One of their key recommendations to overcoming this barrier is using training and joint review of results to educate developers on why the tool is flagging a potential error, what the mitigation options are, etc. Code Integrity has a bunch of deployment and training services to help customers with these types of deployment hurdles.

In our experience, all developers need is one ‘aha’ moment where the tool finds a nasty, subtle bug that would be hard to find using any other method. Once that happens, the developer is a convert. I would also say the burden isn’t just on training, but the tool vendors as well. We all have to continue making the usability of the tool such that developers should be able to instantly recognize why the tool is flagging the error and give the developer all the info they need to recognize the bug and take the appropriate action.

We are starting to see a large amount of Android phones such as the Droid and Xperia X10 (see a review here) and the (soon-to-be-released) first Google phone, Nexus One. With this, expect the number of apps to increase significantly.

Droid vs. iPhone

So with the increased number of apps, do these developers have the right tools to find and fix bugs? Take a look at the leader of phone applications-iPhone. There have been several posts (here and here) that recommend using the Clang static analyzer. Apple has taken it one step further, apparently rejecting iPhone apps that access private APIs. But Clang won’t help you with Java apps.

So what do the Android developers have? Android is just Java, so there are lots of tools, right? Certainly there are static analysis tools, profilers, unit testing tools and many more. But are these tools really taking into account the Android specifics?

Let’s take an example of a resource leak. Resources such as streams, connections and graphic objects must be explicitly closed; otherwise, you run the risk of throwing exceptions depending on the open resource.




For example:

1 static final String propertyFile = "my_config.ini";
2
3 static String getProperyFromConfigFile(String name)throws IOException {
4    Properties prop = new Properties();
5    FileInputStream st = new FileInputStream(propertyFile);
6    prop.load(st);
7    return prop.getProperty(name);
8 }

Here, a resource leak should be identified since line 5 opens up a FileInputStream, but is never closed before exiting the method. Now, this is all well and good and valuable to be found in any Android specific code, but what happens if I’m using built-in classes from the Android SDK?

For example:

1 public boolean onKeyDown(final int keyCode, final KeyEvent event) {
2    if (keyCode == KeyEvent.KEYCODE_DPAD_CENTER) {
3          final MediaPlayer player = MediaPlayer.create(this, ringtoneUri);
4          player.start();
5    }
6    return super.onKeyDown(keyCode, event);
7 }

Here, you have a situation where a MediaPlayer resource is created at line 3, but never closed on exit. Without the knowledge that MediaPlayer is a resource that should be closed, you will miss this type of issue. This extends to many resources and different issues. You can also have Android-specific null pointer exceptions and use of free issues.

Let me know if you’re doing Android development. I want to hear what you are doing to find these kinds of bugs.

I have always been fascinated by the whole area of code vulnerabilities and security exploits and how hackers turn those issues into real-world problems for the rest of us.

Jeremy Brown posted an interesting article on Jeremy’s Computer Security blog where he uses his security know-how to draw a straight line between a software vulnerability found with static analysis and a real 0day exploit on an open source project called gAlan.

Jeremy takes us on a short journey where he finds an unprotected buffer with static analysis, creates an exploit payload to cause a buffer overrun, rewrites the instruction pointer and executes a telnet session, demonstrating how easy it is to turn a run of the mill application into a tunnel into the OS.

One of my colleagues did a similar presentation like this a few years back with a Firefox vulnerability but this is a much better example! Very cool work Jeremy.

Enjoy…

In the first blog series, we discussed the value of compiler warnings and wondered why a static analysis tool would have similar error checking features. In this installment, we want to dive deeper into this question by reviewing errors that can be found by compilers, why they matter, and what limitations compilers have in this area.

Let’s take an example of the “implicit int” rule:

int foo() {
   const x = 0;
   return x;
}

This is a situation where failure to specify a type results in this compiler warning from (gcc v.3.4.4) or Microsoft cl (v.14):

gcc -Wall -c main.c
main.c: In function `foo':
main.c:2: warning: type defaults to `int' in declaration of `x'

cl -c -Wall main.c
main.c(2) : warning C4431: missing type specifier - int assumed. Note: C no longer
supports default-int

You can’t rely on the standard C/C++ implementations to support the implicit int anymore and these compilers alert you to that.  I do have to say, I’ve never seen anyone do this in practice, but it’s nice to know it’s there.

Let’s look at another example:

void foo() {
   if (sizeof(char) < 2)  // defect - the condition is constant
   {
      /* ... */
   }
}

The issue above is that the condition is constant.  See the C99 standard for details on this (section 6.6).  If we run the cl compiler we get:

cl -c -Wall main2.c
main2.c(2) : warning C4127: conditional expression is constant

Here, the cl compiler finds the issue, gcc does not (well, at least my version).   Okay, interesting let’s take a look at a C++ example:

class A
{
   public:
   // non-virtual destructor
   ~A();
   virtual void f1();
};

With this example, if you run either gcc or cl you get the same thing:

gcc -Wall -c main3a.cpp
main3a.cpp:2: warning: `class A' has virtual functions but non-virtual destructor

cl -c -Wall main3a.cpp
main3a.cpp(7) : warning C4265: 'A' : class has virtual functions, but destructor is
not virtual instances of this class may not be destructed correctly

According to the output from both compilers, we made a boneheaded mistake and forgot to assign the destructor as virtual.  Let’s go one step further and define a new method:

void deleteA(A *a) {
   delete a;
}

This method adds a new level of complexity.  When an object of a class derived from the given one is deleted through a pointer to the given class, the destructor of the derived class is not executed, and members of the derived class are not disposed of properly.  In this case, you will not get any warnings from any compiler.  The difference here is that compilers only work within the context of the file/function.  In this case, you are out of luck with compilers, but luckily source code analysis excels in this.

So, the message here is that compiler warnings are quite useful, but they do have their limitations.  Not all compilers report the same things consistently, nor do they cover analysis beyond a single function or file.  Still, make sure you run the compiler warnings, then implement static source code analysis as part of your process to go deeper and find some more complex issues in your code.

For the next blog of this series I’ll cover coding standards and where they fit in your code quality process.