Just wrapped up a successful 2 day Embedded System Engineering conference in Stuttgart, Germany. This “all-German” show had just shy of 600 attendees, as well as about 60 individuals (representing the 20 or so companies exhibiting), so this was considered very good by the show organizers (who by the way did a fantastic job… the food here, for example, was as good as I’ve ever seen for such an event). The Klocwork booth was shared with our good friends at Emenda, and we had a choice spot that allowed a good flow of people. We had an interesting mix at our booth as well… a Scotsman who now lives in Germany and speaks flawless German (albeit with a hint of a wee Scottish accent), an Englishman who had numerous stories that kept us entertained during the quiet times, and myself, the jetlagged Canadian.

As I mentioned earlier, this show is advertised as the only German-language conference around… and it was. So other than saying “hello”, “goodbye”, “thank you”, or “another beer please“, my German is, uhm, lacking. However, not a problem here; the Germans all speak very good English… which was a good thing since my presentation was in English. I had over 40 attendees at my session about how Source Code Analysis fits into Agile development environments, and it went very well. A number of attendees came to our booth after the talk to pick up our White Paper onstatic analysis and agile, and to get a demo of our latest release.

My two-week stint of planes, trains and automobiles continues tomorrow when I head up to Berlin for the weekend to see some good friends (and a football game in the Olympic Stadium), then it is back home on Monday. It has been a great couple of weeks in Europe, but I am looking forward to being back on good ole EST.

I have always been fascinated by the whole area of code vulnerabilities and security exploits and how hackers turn those issues into real-world problems for the rest of us.

Jeremy Brown posted an interesting article on Jeremy’s Computer Security blog where he uses his security know-how to draw a straight line between a software vulnerability found with static analysis and a real 0day exploit on an open source project called gAlan.

Jeremy takes us on a short journey where he finds an unprotected buffer with static analysis, creates an exploit payload to cause a buffer overrun, rewrites the instruction pointer and executes a telnet session, demonstrating how easy it is to turn a run of the mill application into a tunnel into the OS.

One of my colleagues did a similar presentation like this a few years back with a Firefox vulnerability but this is a much better example! Very cool work Jeremy.

Enjoy…

Part I – Ode to Joy

Since the launch of the seminal “Joy” work which hopefully doesn’t need mention here, we’ve seen everything from The Joy of Cooking to The Joy of Not Working (my personal favorite!), and so further to that deeply mined vein of authoritative works we bring you the necessarily over burdened… Joy of Code Review!

Joy, you say? Let me count the ways…

• I implement a task, using what I consider to be best practice patterns and guidelines; I slave over this, my creation, and when it’s done, I stand back and admire, much in the tone of an old master, this latest image of my greatness.
• Then I remember I need to get it reviewed…
• So, I timidly invite my Architect and 3 of his best friends to the war room to review my new baby
• After many rescheduling pauses, we finally gather…
• I hold my breath, turn on the projector, and bare my soul to the collective seniors in attendance
• 30 minutes later, having endured a ritual mind flaying, and the predictable but nevertheless enjoyable tortured examination of my parentage, education, upbringing and such fun rhetorical musings as “why do they let people like you graduate?” I slink out
• Follow up is, if anything, more painful as I’m reminded moment-by-moment of just how badly I’ve lived up to the expectations laid out for me by the senior team members

Anyway, so code reviews suck, amirite? But we all know we need to do them. Of course, we all know we need to do them for completely different reasons from each other…

• Kids right out of grad school know they need to do code reviews because although their code is, like, totally perfect, it’ll be good to show the old dudes their skillz, and for the old dudes to check out some rad new stuff that they might have missed along the way.
• Senior guys know they need to do code review because otherwise all kinds of terrible cruft will get promoted into the head branch and somebody (are you looking at me??) will have to fix it…
• Managers know they need to do code reviews because they read all about them in a book with a cool cover, and it’s all Agile and stuff, and let’s face it they’re being measured on code review coverage, so come hell or high water you’re going to do code reviews!
• And of course, regular professional developers know that code reviews, however painful, genuinely lead to better code, regardless of the pain involved in getting there.

What we have here, folks, is a social organization, complete with the crazy uncle, the embarrassing grandma and the pimply teenagers. And social organizations, as we’ve all come to know and love, are at their best when the forum in which they’re fostered exists for a reason that encourages the unstated, but nevertheless in-your-face activity of which those in the respective societal groups are desperately in need:

• Facebook? Getting a date. And then getting another one while simultaneously trying desperately to avoid the previous partner. Rinse/repeat. Seriously, I have no idea how kids manage today. At least when I was young and awkward we could hide behind the silence and foot shuffling of real face-to-face meetings. Now with a keyboard and the internet in the way, there’s nowhere to hide!!!! I’m off topic again… ahem…
• Linked-In? Getting a job.
• Myspace? Getting a clue.

You get the idea.

So code review as a social engagement… really? Parts 2 and 3 of this series of posts will examine how such interactions, fostered by social networking tools, are the best way to ensure code review gets done and returns value both to the participants and to the companies in which they work.

In the first blog series, we discussed the value of compiler warnings and wondered why a static analysis tool would have similar error checking features. In this installment, we want to dive deeper into this question by reviewing errors that can be found by compilers, why they matter, and what limitations compilers have in this area.

Let’s take an example of the “implicit int” rule:

int foo() {
   const x = 0;
   return x;
}

This is a situation where failure to specify a type results in this compiler warning from (gcc v.3.4.4) or Microsoft cl (v.14):

gcc -Wall -c main.c
main.c: In function `foo':
main.c:2: warning: type defaults to `int' in declaration of `x'

cl -c -Wall main.c
main.c(2) : warning C4431: missing type specifier - int assumed. Note: C no longer
supports default-int

You can’t rely on the standard C/C++ implementations to support the implicit int anymore and these compilers alert you to that.  I do have to say, I’ve never seen anyone do this in practice, but it’s nice to know it’s there.

Let’s look at another example:

void foo() {
   if (sizeof(char) < 2)  // defect - the condition is constant
   {
      /* ... */
   }
}

The issue above is that the condition is constant.  See the C99 standard for details on this (section 6.6).  If we run the cl compiler we get:

cl -c -Wall main2.c
main2.c(2) : warning C4127: conditional expression is constant

Here, the cl compiler finds the issue, gcc does not (well, at least my version).   Okay, interesting let’s take a look at a C++ example:

class A
{
   public:
   // non-virtual destructor
   ~A();
   virtual void f1();
};

With this example, if you run either gcc or cl you get the same thing:

gcc -Wall -c main3a.cpp
main3a.cpp:2: warning: `class A' has virtual functions but non-virtual destructor

cl -c -Wall main3a.cpp
main3a.cpp(7) : warning C4265: 'A' : class has virtual functions, but destructor is
not virtual instances of this class may not be destructed correctly

According to the output from both compilers, we made a boneheaded mistake and forgot to assign the destructor as virtual.  Let’s go one step further and define a new method:

void deleteA(A *a) {
   delete a;
}

This method adds a new level of complexity.  When an object of a class derived from the given one is deleted through a pointer to the given class, the destructor of the derived class is not executed, and members of the derived class are not disposed of properly.  In this case, you will not get any warnings from any compiler.  The difference here is that compilers only work within the context of the file/function.  In this case, you are out of luck with compilers, but luckily source code analysis excels in this.

So, the message here is that compiler warnings are quite useful, but they do have their limitations.  Not all compilers report the same things consistently, nor do they cover analysis beyond a single function or file.  Still, make sure you run the compiler warnings, then implement static source code analysis as part of your process to go deeper and find some more complex issues in your code.

For the next blog of this series I’ll cover coding standards and where they fit in your code quality process.

In a previous posts I reviewed the Top 5 C/C++ and Top 5 C# quality bugs that I that I see time and time again looking at customer code. I wrote my Java Top 5 with an embedded programming focus and the folks at www.embedded.com decided to publish it on their site. Here’s a snippet below and the full Top 5 Java bugs article can be found here.

While C dominates as the programming language of choice for embedded development, the use of Java is definitely on the rise. In fact, according to a recent VDC survey, 12.3% of respondents currently use Java in the embedded space, and 17.9% expect to be using Java in the next two years.

For those transitioning from embedded development using C, you might find yourself falling into the hype that Java is a “safe” language. For example, Java developers face no requirement for managing memory associated with objects. However, this is where the trap may be laid. Even though there’s no need for memory management, developers may need to keep track of specific resources the object allocates. This is especially true in an embedded context where resources are often constrained. Even for experienced developers, these traps pop up time and again and can easily jeopardize your code quality and security.

Here’s a round-up of the top five programming issues developers should be aware of in embedded Java development [More...]

In this 3 part blog series I want to cover general misconceptions with static analysis coverage.  This will include a discussion about:

• compiler warnings available,
• different types of style issues including coding standards, and
• your available options to fit them into your formal process.

Very often customers ask why we don’t cover specific checkers.  We always get great feedback on high value checkers that they would like to see.  But occasionally we get the request to find simple compiler warnings or code style issues.

For the first part of this series I want to focus on compiler warnings.  These are not the compiler errors such as syntax/parse errors you get with a compiler.  Instead, I want to concentrate on those pesky compiler warnings that still let you build your application, when you really know you shouldn’t.  We have all experienced compiler warnings that are just plain confused with your code.  But on the whole, you are guaranteed to find some pretty big blunders.

Most modern compilers focus on providing more details about compiler warnings.  These can be very valuable as it helps find many of those plain dumb mistakes.  It varies by compilers, but many find things such as constant expressions from conditionals, returning from a void function, assignment in condition (use = instead of ==), suspiciously-placed semi-colon and many, many more.

To find some of the issues, you usually need to provide a compiler flag -Wall.  For example:

    gcc -c -Wall foo.c 

Make sure you read your compiler documentation for available warnings.  Here is the gnu gcc compiler and Microsoft cl compiler docs.

Given that every compiler on the market provides its own “checkers” for compiler issues, does it really make sense for static analysis to get in there and detect these issues again?  I strongly believe that every developer should ALWAYS clean up their compiler warnings before going onto static analysis.

But you will still find static analysis tools providing these capabilities.  Why?  Well, first and foremost, not every compiler has the capability to find simple coding issues.  The other reality is that not everyone checks the compiler warnings…(we all know who we are).  Or sometimes you just want to run one tool.  In other words get the more complex bugs with static analysis along with the compiler warnings.  It is for these reasons that static analysis tools have introduced many of these low-level issues.

For the next part of this blog series, I want to go into the details of compiler warnings and some of the things that coding standards are doing as well.

The first instalment in my ‘Going Agile’ series will reflect on the earliest days that led up to our development team becoming an Agile development team. Before I get into this too deep, I should first set the stage a little. This organization is a medium sized ($500M in revenue) software company, with no other teams using Agile techniques. We were going to be the first. The product was well established, having been on the market for about 5 years, and traditional development methods were fairly effective from a delivery and quality perspective. The team comprised of about 12 members, which included all developers, testers, documentation, the Scrum Master, and myself as the Product Owner.

The first meeting ever held with the whole team was intended to introduce Agile (Scrum) to them and get their buy in…we did not want to ram it down their throats and say we are doing this. At the time we were doing this, the concept of an Agile Coach was pretty much non-existent, so we were on our own. Our Scrum Master had been on some training, and thought for this meeting it would be good to try an exercise with the team to illustrate how Agile could work. So without any preamble or mention of Agile, everyone in the room was instructed to pick a partner, and then the goal of the exercise was explained. The goal was to navigate your team member around the room as many times as possible in 1 minute. Seemed pretty easy, until we told them about the twist…the team member that was walking around the room was to be blind folded. To add to the fun, our Scrum Master played the role of a roadblock, stepping in front of the blind folded people as they tried to circle the room, requiring new instructions to be shouted out to navigate around him. Once the clock started the chaos broke out, as 6 different people were simultaneously shouting out instructions trying to navigate their team member around the room, and around the roadblock. By the end of the 60 seconds we tallied total trips around the room, and came up with a number in the 30’s.

The 2^nd part of the exercise was quite a bit simpler, and merely had all team members walk around the room (no blind folds), and make their own decisions to navigate around the room and the ‘roadblock’. I’m sure you can imagine how much calmer the room was, and by the end of the 60 seconds we again tallied our results. This time we were well over 120.

So what was the purpose of this exercise, and how did it relate to Agile? Basically it illustrated the power of a self-managing team, and how the productivity of such a team can be magnitudes better when team members are given the ability to make decisions on their own, in real-time. After the exercise, we then introduced the idea of the team adopting Agile, what it is, how it could work, etc. In my opinion, doing this exercise first (and doing it without ever mentioning the term Agile) did more than a week’s worth of powerpoint sessions on why Agile is the way to go. The team was very receptive to the idea (at least most members, more on that in another blog), which set the stage for a series of follow on meetings to further flesh out what was really involved, and what needed to be done to prepare for our first Iteration. The next instalment in this series will delve into that preparation.

Good first day at ESC Boston2009. Gwyn and Alen presented a well attended talk on using source code analysis (SCA) to improve developer productivity. Key takeaways from the presentation:

• How SCA will impact your development velocity
• Quick history on SCA – talked about lint and the general evolution of the technology
• How the information generated by static code analysis can be used to solve a variety of development challenge
• Demo of where SCA fits from a code review, refactoring and bug detection standpoint

Interesting change from past presentations where most people now understand the basics of the technology… no need to spend too much time talking about its history and technology building blocks.

The presentation was recorded so we’ll load the video up at a later date for everyone’s viewing pleasure :)

In a previous post I provided the top 5 C/C++ quality bugs that I that I see time and time again looking at customer code.  Time for the C# version:

1.    Null pointer exceptions from a method

1                  public class A {
2                      public A foo() {
3                          if (flag)
4                              return null;
5                          return new A();
6                      }
7
8                      public void var() {
9                          A a = foo();
10                         if (flag)
11                             a.foo();
12                     }
13
14                     private int flag;
15                 }

This is the most common issue I see.  In this example a warning is issued at line 11 for a possible null pointer exception.  Essentially there is a potential for a null value from the method foo().  I believe I see these issues more than any others  because of the inter-procedural context of this issue.  This example is quite simple, but where you can easily get lost is when that null value is coming from a long method call chain in multiple classes.

2.    Resource leaks

1  using System;
2  using System.IO;
3
4  namespace LeakExample
5  {
6      class Test
7      {
8          public String Run(String name)
9          {
10             StreamReader reader = new StreamReader(name);
11             String result = reader.ReadToEnd();
12             reader.Close();
13             return result;
14         }
15     }
16 }

I think for most of us this is pretty easy to see.   An object of ‘StreamReader’ type is allocated and its reference is assigned to the ‘reader’ member. If the call to ‘ReadToEnd’ throws an exception, control is transferred out of method ‘Run’, variable ‘reader’ goes out of scope, the object referenced by it becomes lost, but related resources are not disposed.  See my previous blog post on how some people miss this and what you need to do to fix this.

3.    Forward null pointer exception

1                  public class A {
2                      public void foo() {
3                          A a = new A();
4                          if (a == 0)
5                              if (flag)
6                                  a.foo();
7                      }
8                      private int flag;
9                  }

This is another example of a null pointer exception with a twist.  In this example the class data member ‘a’ is compared with 0 value at line 4, and therefore may still be expected to be null when it is dereferenced at line 6.

4.    Reverse null pointer exception

1                  public class A {
2                      public void foo() {
3                          A a = null;
4                          a.foo();
5                          if (a == null)
6                              a = new A();
7                      }
8                  }

Yes, another null pointer exception (getting a theme here?).  Same idea at #3 but in reverse order.  Here they dereference the data member ‘a’ at line 4 but later check for null at line 5.

5.    Empty catch clause

1  class FileHandler {
2      public void Open(String name) {
3          try {
4              // opening file ...
5          } catch (FileNotFoundException e) {   // defect - no statements in the 'catch' clause
6          }
7      }
8  }

Okay, so not exactly the big impact as the top 4, but I just had to mention it.  I’m always guaranteed to see some of these.  Granted they are maintainability issues more than anything else, but c’mon they are there for a purpose.
As promised, I did say that I would post the Java version soon.  It is next…

Truepath Analysis

Ever been faced with that glassy-eyed expression, the look of unthinking, unwholesome fear when some long, incomprehensible word escapes your geeky mouth and upsets the maiden aunts around the once-a-year, wear-your-best-tie, try-not-to-die-before-the-cake’s-all-gone tea table? OK, so this paper won’t help you in that situation whatsoever, but if you replace your maiden aunts with a bunch of your best geek friends, and replace the tea with a sturdy helping of Dew, knowing how a real whole program analysis solution works might just conceivably come in handy. Some day. “Dude, I was totally stoked when I read this thing, trust me it’s ahh-some.” Maybe.

Anyway, in the best traditions of self-serving corporate PR blogs everywhere, I give you… drum roll please… Whole Program Analysis, the Klocwork Way.

Enjoy.