There was a recent article from Mark Pitchford titled: “Think static analysis cures all ills? Think again.” Obviously being biased working here at Klocwork, I take a major exception to what Mark has to say.

This article makes ridiculous claims. About the only thing Mark got right was that static analysis has been around for a long time. However it’s ludicrous to think that they’re the same as they were in the past. That’s like saying computers from decades ago are the same as today. The advancement has been huge for static analysis tools, especially in the last couple of years.

The author is really selling the merits of dynamic testing, which is great. Everyone should have the proper testing procedures in place. But static analysis is complementary–it’s another tool that will help you go through ALL the paths of your code to help you find bugs you’d otherwise miss.

One of the biggest reasons static analysis tools have taken off, in my opinion, is the level of integration. It’s quite simple to get results with static analysis tools with little effort. Especially compared with dynamic analysis tools.

Paul Anderson, a fellow competitor, sums it up very nicely in the comments. Check it out.

Static Analysis Tools Not Created Equal

Yes, it’s true (!) and as anyone in this space knows there is a huge difference between static analysis tools, their level of sophistication, and their approach to developer adoption. Gary McGraw & John Steven from Cigital describe their views on this topic including ‘5 pitfalls’ that customers should avoid when evaluating tools. These pitfalls mostly amount to the fact that analysis results across different tools, code bases, and tool operators can make results vary significantly, so be aware of this fact when conducting your benchmarking. Their overall recommendation:

“The upshot? Use your own code instead of a pre-fab evaluation suite. You probably have the makings of a good set of tests within your own organization’s application base….”

I agree with this and can honestly say we rarely, if ever, run into evaluations where customers exclusively use pre-fab test suites instead of their own code for many of the reasons outlined in their article. So, I’d say the market is (and has been for some time) embracing this recommendation wholeheartedly. So, beyond this recommendation, what else should customers consider when evaluating these tools? Here are a couple other significant areas to consider where you’ll find that, yes, all tools aren’t created equal.

• Environment support. In particular in the embedded software space, considerations such as integration with your build environments, compiler support, ability to work with multiple software branches, are all crucial considerations for a successful deployment. Not all tools have good support in these areas, but these capabilities can often make or break a deployment.
• Developer adoption. This is everything frankly, and a big part of achieving developer adoption is the quality of the analysis issues raised in the article. Obviously, a tool that generates accurate, useful results will get you well on your way to strong developer adoption, but that’s not everything. How are the defects described to developers, including the trace info? Do developers want to run their own desktop static analysis rather than fetching results periodically from the integration build? If so, how smart is the vendor’s desktop analysis?

So basically, picking a tool boils down to assessing: quality & flexibility of the analysis, support for your dev environment (not just the one you’re using in the eval!), and thinking ahead to developer adoption issues. Assess these three areas thoroughly and you’ll end-up picking the tool that’s right for your needs.

It happened again.  For what seems like the 100th time, someone reports to me that they are seeing a number of false positive reports on the resource leak checker.  For those not familiar with a resource leak, take a look at a previous post.  Although resource leaks apply across most languages, the place where this question keeps coming  up seems to always be in Java or C# code.  My last query came from Java code, so we will use that as an example.  Here was a report where the FileInputSteam is not closed on exit.

FileChannel sch = new FileInputStream(...);
FileChannel dch = new FileOutputStream(...);
try{
   ...
} catch (...) {
   ...
} finally {
   try {
      sch.close();
   }
}
...

Do you see this issue?  Clearly, they close the FileInputStream in the finally block.  The problem is that you can still throw an exception when you try to create a FileOutputStream.  The simple fix is to encompass the FileInputStream and FileOutputStream in the try/catch blocks.

Static analysis tools are great for finding bugs like this resource leak.  In this case, the report that a FileInputStream is not closed on exit wasn’t enough information to debug this.  This brings us to a very important discussion with static analysis tools–trace.

Gone are the days where a static analysis tool reports a bug at line X with no context.  Today you get detailed traces.  The trace contains conditions and assignments to give you clues about why the tool is identifying a bug.  In this example below, identifying the “source” and “sink” of any defect will help you trace through the start/end of the control flow.

More importantly in the case of Java, here we have a clear understanding where exceptions are thrown.  This would have helped tremendously for the example above.

Detailed traces are very important and can do many things for individual checkers.  So whatever you do, don’t just look at the bug description, pay attention to the trace!

The two sides of testing team lead Jonathan Patchell.

At daily standup meetings, they eye each other from opposite sides of the room. Sitting on the same side of the cubicle wall is unthinkable.

They’re united only by their desire to produce quality software products and their appreciation for coffee and energy drinks. What’s good to one side can be anathema to the other when it comes to code.

I’m talking, of course, about testing and development teams. In the interests of generating more comments improving dialogue between two very important functions in a software organization, our marketing director asked me to interview our testing team lead, Jonathan Patchell, about the ways in which developers drive his team nuts.

Patchell, a computer systems engineer, has been with Klocwork for five years and a team lead for two. He struck a fairly conciliatory tone for this interview, which sorta ruins the adversarial approach, but don’t let his diplomacy fool you. I’ve seen him suffering as the release date approaches and his demeanour changes completely.

Here are Patchell’s top dev peeves:

As a product manager, the only backlog I typically care about is my product backlog. Do I have the right stories in there? Do the stories have enough detail? Are they properly prioritized? You know, that kind of stuff. Today, however, I’m going to write about a very different backlog, that is the static analysis defect backlog.

A static analysis backlog is created when you run a static analysis product on your code base for the very first time. Chances are pretty good that the first analysis is going to list a large number of defects, some that are without question real, and some that perhaps are not. Do not freak out! This is the first time that analysis engine has ‘laid eyes’ upon your code and it is going to flex its muscles and show you any weaknesses it believes exist. So how does one deal with this? Here are a few strategies to help you:

1) Don’t boil the ocean. Before you even run that first analysis, don’t have a “wouldn’t it be cool” moment, where you decide to turn on every single rule the analysis engine has. There is a reason why static analysis tools haven’t turned on everything.  They are showing the most accurate and critical issues first.  So unless you have unlimited time and resources, your best bet is to start with a core set of rules and run the analysis based on that set. This core set of rules should include things such as memory/resource leaks, buffer overruns, null pointer dereferences, uninitialized variables, and so on. Add other rules once you have this core set under control.

Is your issue backlog making you cross eyed? Try these coping strategies.

2) Baseline your defects. Consider that first analysis your baseline and choose to ‘park’ them for the time being. Chances are the product that the analysis was run on is one that has already been released to the public, and in good working order. Zero out these defects for now, and start to triage them, which leads into strategy #3.

3) This is going to sound pretty obvious, but when it comes to managing your issue backlog start looking at the most critical issues first. These are the ones that are most likely to cause a failure of some sort, so determine if these issues are real, and if so, fix them immediately. Once you’re done with the most critical issues, move to the next level of severity, and continue on that way.

4) Finally, tune your analysis. Any good vendor will allow you to tune your analysis. The benefits of tuning are twofold; 1) you can find code issues that would otherwise go undetected and, 2) reduce the number of issues that the engine reports incorrectly in the context of your source code. You should think of ways to give the tool more context about your code base to increase accuracy.

If you follow these suggestions, you’ll definitely have a better grasp of your bug backlog, and you’ll be able to execute on reducing that backlog quickly and efficiently. If you don’t, then at some point, you may feel a little like the critter pictured here.

If there are any other strategies you’ve tried to deal with your bug backlog, leave a comment or two. I’d love to hear about them.

The year 2008 was a key year for mobile applications. In that year, Apple released its iOS SDK in March and launched the App Store with the release of iOS 2.0 in July. Let’s call it the start of the Mobile Gold Rush. Now in this mobile gold rush, there are hundreds of thousands of applications and amongst them many are bound to have the same idea and the same purpose. How does one app shine, while others won’t even get visits to their description pages?

Let me tell you about an experience I had. I used to own a smartphone running Windows Mobile 6.1. I loved the phone when I only used it as a phone, but simply hated it when it came to applications. There were thousands of issues I could have pointed out. The end result is that I am not going to purchase another Windows smartphone. Do you see where I am going with this?

Consumers always rely on their memory associations, whether conscious or unconscious, when it comes to purchasing new products. I would say almost everyone would not go back to using a product they’ve had a bad experience with when there are so many other options around. This goes the same for mobile application developers and development firms. I have uninstalled so many applications from my Nexus One within the first few minutes of their lives. It wasn’t because of the features they didn’t have or how horrid the GUI was. The main reason was they weren’t working the way they were expected to. Some users even kiss applications goodbye altogether according to this survey based on such experiences. Let me put it this way:

“If your code is not flawless, you will lose your market share and never be able to recover it.”

Application developers strive to develop new features giving them the competitive advantage or as my friend and mentor, Bruce Firestone, calls it “Pixie Dust”. This is completely the right thing to do; however, they should focus more on their apps’ perfect functional execution. Having a limited number of features that work exactly as the user expects is better than having more numerous, but buggy features. I know it sounds like a no brainer, but the success of a small number of apps as opposed to the thousand other ones doing the same thing should serve as sufficient evidence that it is easier said than done.

Buggy code hurts the application and the developing company’s brand. Making sure your code is near perfect would be a strategy I would like to call the Pre-Branding Protection Plan. With the abundance of competitors in the mobile gold rush, bad apps will almost permanently prevent market recovery and destroy sales.

One method I use to make sure my brand would be protected is using J2ME static analysis tools. There are various paid and free tools, but I am very happy with the Klocwork Solo, which is geared for J2ME developers. I had never used such tools and only started using them when I joined the company. I don’t know what I would do without them now. In my next posts, I will discuss some of the issues the tool caught that improved my productivity and the efficiency of my code.

After six years of post-secondary education, my first day of the real world had finally come.   As I approached the doors to Klocwork, I realized it was time to put all my years of education to the test.

Straight out of high school, I had little idea of what career path I should take. Four years of university passed and I graduated with a B.A. in English, but still, I didn’t feel prepared. Another two years of college in media-related studies and, ready or not, it was time to make the leap into the working world.

My first day could be described in exactly two words: a blur. Between getting set up at my desk and meetings with my superiors, it was easy to feel overwhelmed. Nevertheless, after I got past the initial nerves of starting a new job, I felt ready for whatever was to come my way.

As it was put in my very first meeting at the company, I would be given “a taste of everything”. A taste was exactly what I had needed to get myself started and get some experience under my belt.

Fast forward almost two months and the experience has been nothing short of a rollercoaster.

One of my most rewarding and yet most challenging experiences since I started here has been a project which consisted of redesigning part of the documentation wiki. One of my colleagues, Patti, has been in process of designing a new landing page, and I had been asked to carry her design over to the wiki. While the task might seem simple, it has been challenging.

When I first glanced at the page, it didn’t think there would be many issues. I’m not afraid to admit that I may have made a poor assumption.

One revision after another, I faced one obstacle after another. Problems ranged from something as simple as getting the correct spacing to more complex issues dealing with specific browsers. The end of the week was near and I felt as if I was getting nowhere.

I let myself walk away from it for the weekend and came in the following week with a refreshed mindset. While it was quickly becoming evident this particular design would not work, there was still light at the end of the tunnel. With some guidance from the documentation team, we quickly got back on track.

Now with the page’s design nearing completion with approval from the key players, it’s something I certainly feel proud of. It’s a prime example of how much I’ve learned in the short time I have been with Klocwork.

While I know work may not always be fun (a few repetitive tasks come to mind), at the end of the day, I think it is projects like this that make it all worthwhile.

In my next post, I’ll share some more of my experiences working with the support, documentation and training teams.

If you’ve ever gone through the process of porting an application, you know the pain.  Porting can be difficult if you’re not vigilant from the outset.  There are tons of written guidelines and best practices for specific platforms or architectures, such as those going to 64 bit for Windows apps or Intel architecture and Mac OS.

In the past, we have talked about Endian issues, which are very specific to porting from different architectures (big-endian vs little-endian).  This time I want to take you through some general porting issues to show you how you can now use static analysis tools to help you through this process.

A very common gotcha are situations in which an expression is cast to a type of potentially different size.  Code written for a particular architecture’s data model can face significant challenges in porting when the new architecture imposes a new data model. For example, when porting between 32 bit and 64 bit data models, the new data model typically leaves ‘int’ at 32 bits, but uses ‘long’ as a 64 bit value, breaking any assumed equality in data type size that worked under the 32 bit data model.  Using static analysis tools can help you find this automatically.  In the example below all casts will throw an error:

void bad(){
 unsigned char uc = 0;
 unsigned short us = 0;
 unsigned int ui = 0;
 unsigned long long ull = 0;
 long l=0;

 l = (long)ull; //porting issue
 uc = (unsigned char)l; //porting issue
 us = (unsigned short)l; //porting issue
 ui = (unsigned int)l; //porting issue
}

Another gotcha is using bit fields within a structure.  When you define a data structure using bit fields, you are relying on the endian representation of a particular compiler. Moving to a different compiler can modify this layout, resulting in problems when that data structure is projected onto a byte buffer.  Again static analysis tools can be used to identify where the bit field definition exists and NOT properly covered by a big-endian or little-endian macro.  For example flag this:

struct mystruct{
   unsigned short x : 4;
   unsigned short y : 5;
   unsigned short z : 7;
} test;

But don’t flag the same thing with conditional compilation:

#ifdef BIG_ENDIAN
struct mystruct{
   unsigned short x : 4;
   unsigned short y : 5;
   unsigned short z : 7;
} test;
...

There are tons of other gotchas as well, such as using a custom byte swap macro without checking endianness, an incompatible type is used with a network macro such as ‘ntohl’, compiler-dependent option is used, macro describing a builtin numeric type is used, and many more.

Using static analysis tools not only detects porting issues, but provides metrics and reporting capabilities that allow you to prioritize and identify the scope of work involved.  For example, Top 10 porting issues (see the diagram above) helps you identify the bulk of the issues that need to be addressed.  Or you can use the component diagram report to identify specific components of your software code base you may have to concentrate on going forward.

Nice blog post from Walter Bright over at Dr. Dobbs on the Patterns of Bugs. He ties together bug patterns, recommended process changes, and the resulting productivity payoff from making these improvements. He recommends a bunch of process changes, including static analysis, code reviews, and coding standards, then goes on to review examples of different bug patterns. A few can be detected with static analysis (coding mistakes as written) but many are errors with the code as intended (something static analysis doesn’t check… that’s what testing is for). His main recommendation seems to be that bugs can often be pattern based, so once a bug is identified, take steps to remove that pattern from your code through process or tool changes.

Once the pattern is identified, then look for changes in process that will permanently eliminate that bug. Eliminate enough of the bug patterns, and you should enjoy a substantial increase in productivity.

His experience is that, over time this kind of systematic approach to fixing bugs makes developers better.

I’ve noticed in my decades of writing programs that I just don’t make the kinds of mistakes I used to. Apparently I’ve unconsciously evolved coping strategies to avoid them. Identifying and building such strategies into the process means everyone can benefit from that experience.

We certainly see that in the area of static analysis bug detection; lots of customers report that their developers make fewer mistakes, defect injection rates go down, and overall productivity is improved.

Yesterday we did our second Medical Devices software seminar, this time in snowy and cold Minneapolis. Say what you will about the weather, but this city is built for winter…it has various overhead ‘tunnels’ called ‘skyways‘ connecting what seemed to be the entire downtown core, so you rarely ever need to go outside.

Anyways, our seminar drew the interest of over 75% of registrants, mostly software engineers and QA, so really another great turnout. The format was the same as our Boston event, with the same players from SterlingTech, Klocwork (duh) and Vector Software. There really didn’t seem to be too much separating this group from the Boston group with the exception being around the development methodologies being used. The Boston group all appeared to be using (or moving to) and Agile methodology, whereas this group was not using Agile at all. For the life of me I cannot explain this. I would have thought if Medical device organizations in one part of the country were primarily using Agile, perhaps it was becoming the norm for that market segment…but based on this sample, now I just don’t know. If anyone can share their observations on this, I would love to hear from you.