There was a recent article from Mark Pitchford titled: “Think static analysis cures all ills? Think again.” Obviously being biased working here at Klocwork, I take a major exception to what Mark has to say.

This article makes ridiculous claims. About the only thing Mark got right was that static analysis has been around for a long time. However it’s ludicrous to think that they’re the same as they were in the past. That’s like saying computers from decades ago are the same as today. The advancement has been huge for static analysis tools, especially in the last couple of years.

The author is really selling the merits of dynamic testing, which is great. Everyone should have the proper testing procedures in place. But static analysis is complementary–it’s another tool that will help you go through ALL the paths of your code to help you find bugs you’d otherwise miss.

One of the biggest reasons static analysis tools have taken off, in my opinion, is the level of integration. It’s quite simple to get results with static analysis tools with little effort. Especially compared with dynamic analysis tools.

Paul Anderson, a fellow competitor, sums it up very nicely in the comments. Check it out.

Static Analysis Tools Not Created Equal

Yes, it’s true (!) and as anyone in this space knows there is a huge difference between static analysis tools, their level of sophistication, and their approach to developer adoption. Gary McGraw & John Steven from Cigital describe their views on this topic including ‘5 pitfalls’ that customers should avoid when evaluating tools. These pitfalls mostly amount to the fact that analysis results across different tools, code bases, and tool operators can make results vary significantly, so be aware of this fact when conducting your benchmarking. Their overall recommendation:

“The upshot? Use your own code instead of a pre-fab evaluation suite. You probably have the makings of a good set of tests within your own organization’s application base….”

I agree with this and can honestly say we rarely, if ever, run into evaluations where customers exclusively use pre-fab test suites instead of their own code for many of the reasons outlined in their article. So, I’d say the market is (and has been for some time) embracing this recommendation wholeheartedly. So, beyond this recommendation, what else should customers consider when evaluating these tools? Here are a couple other significant areas to consider where you’ll find that, yes, all tools aren’t created equal.

• Environment support. In particular in the embedded software space, considerations such as integration with your build environments, compiler support, ability to work with multiple software branches, are all crucial considerations for a successful deployment. Not all tools have good support in these areas, but these capabilities can often make or break a deployment.
• Developer adoption. This is everything frankly, and a big part of achieving developer adoption is the quality of the analysis issues raised in the article. Obviously, a tool that generates accurate, useful results will get you well on your way to strong developer adoption, but that’s not everything. How are the defects described to developers, including the trace info? Do developers want to run their own desktop static analysis rather than fetching results periodically from the integration build? If so, how smart is the vendor’s desktop analysis?

So basically, picking a tool boils down to assessing: quality & flexibility of the analysis, support for your dev environment (not just the one you’re using in the eval!), and thinking ahead to developer adoption issues. Assess these three areas thoroughly and you’ll end-up picking the tool that’s right for your needs.

The two sides of testing team lead Jonathan Patchell.

At daily standup meetings, they eye each other from opposite sides of the room. Sitting on the same side of the cubicle wall is unthinkable.

They’re united only by their desire to produce quality software products and their appreciation for coffee and energy drinks. What’s good to one side can be anathema to the other when it comes to code.

I’m talking, of course, about testing and development teams. In the interests of generating more comments improving dialogue between two very important functions in a software organization, our marketing director asked me to interview our testing team lead, Jonathan Patchell, about the ways in which developers drive his team nuts.

Patchell, a computer systems engineer, has been with Klocwork for five years and a team lead for two. He struck a fairly conciliatory tone for this interview, which sorta ruins the adversarial approach, but don’t let his diplomacy fool you. I’ve seen him suffering as the release date approaches and his demeanour changes completely.

Here are Patchell’s top dev peeves:

As a product manager, the only backlog I typically care about is my product backlog. Do I have the right stories in there? Do the stories have enough detail? Are they properly prioritized? You know, that kind of stuff. Today, however, I’m going to write about a very different backlog, that is the static analysis defect backlog.

A static analysis backlog is created when you run a static analysis product on your code base for the very first time. Chances are pretty good that the first analysis is going to list a large number of defects, some that are without question real, and some that perhaps are not. Do not freak out! This is the first time that analysis engine has ‘laid eyes’ upon your code and it is going to flex its muscles and show you any weaknesses it believes exist. So how does one deal with this? Here are a few strategies to help you:

1) Don’t boil the ocean. Before you even run that first analysis, don’t have a “wouldn’t it be cool” moment, where you decide to turn on every single rule the analysis engine has. There is a reason why static analysis tools haven’t turned on everything.  They are showing the most accurate and critical issues first.  So unless you have unlimited time and resources, your best bet is to start with a core set of rules and run the analysis based on that set. This core set of rules should include things such as memory/resource leaks, buffer overruns, null pointer dereferences, uninitialized variables, and so on. Add other rules once you have this core set under control.

Is your issue backlog making you cross eyed? Try these coping strategies.

2) Baseline your defects. Consider that first analysis your baseline and choose to ‘park’ them for the time being. Chances are the product that the analysis was run on is one that has already been released to the public, and in good working order. Zero out these defects for now, and start to triage them, which leads into strategy #3.

3) This is going to sound pretty obvious, but when it comes to managing your issue backlog start looking at the most critical issues first. These are the ones that are most likely to cause a failure of some sort, so determine if these issues are real, and if so, fix them immediately. Once you’re done with the most critical issues, move to the next level of severity, and continue on that way.

4) Finally, tune your analysis. Any good vendor will allow you to tune your analysis. The benefits of tuning are twofold; 1) you can find code issues that would otherwise go undetected and, 2) reduce the number of issues that the engine reports incorrectly in the context of your source code. You should think of ways to give the tool more context about your code base to increase accuracy.

If you follow these suggestions, you’ll definitely have a better grasp of your bug backlog, and you’ll be able to execute on reducing that backlog quickly and efficiently. If you don’t, then at some point, you may feel a little like the critter pictured here.

If there are any other strategies you’ve tried to deal with your bug backlog, leave a comment or two. I’d love to hear about them.

The year 2008 was a key year for mobile applications. In that year, Apple released its iOS SDK in March and launched the App Store with the release of iOS 2.0 in July. Let’s call it the start of the Mobile Gold Rush. Now in this mobile gold rush, there are hundreds of thousands of applications and amongst them many are bound to have the same idea and the same purpose. How does one app shine, while others won’t even get visits to their description pages?

Let me tell you about an experience I had. I used to own a smartphone running Windows Mobile 6.1. I loved the phone when I only used it as a phone, but simply hated it when it came to applications. There were thousands of issues I could have pointed out. The end result is that I am not going to purchase another Windows smartphone. Do you see where I am going with this?

Consumers always rely on their memory associations, whether conscious or unconscious, when it comes to purchasing new products. I would say almost everyone would not go back to using a product they’ve had a bad experience with when there are so many other options around. This goes the same for mobile application developers and development firms. I have uninstalled so many applications from my Nexus One within the first few minutes of their lives. It wasn’t because of the features they didn’t have or how horrid the GUI was. The main reason was they weren’t working the way they were expected to. Some users even kiss applications goodbye altogether according to this survey based on such experiences. Let me put it this way:

“If your code is not flawless, you will lose your market share and never be able to recover it.”

Application developers strive to develop new features giving them the competitive advantage or as my friend and mentor, Bruce Firestone, calls it “Pixie Dust”. This is completely the right thing to do; however, they should focus more on their apps’ perfect functional execution. Having a limited number of features that work exactly as the user expects is better than having more numerous, but buggy features. I know it sounds like a no brainer, but the success of a small number of apps as opposed to the thousand other ones doing the same thing should serve as sufficient evidence that it is easier said than done.

Buggy code hurts the application and the developing company’s brand. Making sure your code is near perfect would be a strategy I would like to call the Pre-Branding Protection Plan. With the abundance of competitors in the mobile gold rush, bad apps will almost permanently prevent market recovery and destroy sales.

One method I use to make sure my brand would be protected is using J2ME static analysis tools. There are various paid and free tools, but I am very happy with the Klocwork Solo, which is geared for J2ME developers. I had never used such tools and only started using them when I joined the company. I don’t know what I would do without them now. In my next posts, I will discuss some of the issues the tool caught that improved my productivity and the efficiency of my code.

If you’ve ever gone through the process of porting an application, you know the pain.  Porting can be difficult if you’re not vigilant from the outset.  There are tons of written guidelines and best practices for specific platforms or architectures, such as those going to 64 bit for Windows apps or Intel architecture and Mac OS.

In the past, we have talked about Endian issues, which are very specific to porting from different architectures (big-endian vs little-endian).  This time I want to take you through some general porting issues to show you how you can now use static analysis tools to help you through this process.

A very common gotcha are situations in which an expression is cast to a type of potentially different size.  Code written for a particular architecture’s data model can face significant challenges in porting when the new architecture imposes a new data model. For example, when porting between 32 bit and 64 bit data models, the new data model typically leaves ‘int’ at 32 bits, but uses ‘long’ as a 64 bit value, breaking any assumed equality in data type size that worked under the 32 bit data model.  Using static analysis tools can help you find this automatically.  In the example below all casts will throw an error:

void bad(){
 unsigned char uc = 0;
 unsigned short us = 0;
 unsigned int ui = 0;
 unsigned long long ull = 0;
 long l=0;

 l = (long)ull; //porting issue
 uc = (unsigned char)l; //porting issue
 us = (unsigned short)l; //porting issue
 ui = (unsigned int)l; //porting issue
}

Another gotcha is using bit fields within a structure.  When you define a data structure using bit fields, you are relying on the endian representation of a particular compiler. Moving to a different compiler can modify this layout, resulting in problems when that data structure is projected onto a byte buffer.  Again static analysis tools can be used to identify where the bit field definition exists and NOT properly covered by a big-endian or little-endian macro.  For example flag this:

struct mystruct{
   unsigned short x : 4;
   unsigned short y : 5;
   unsigned short z : 7;
} test;

But don’t flag the same thing with conditional compilation:

#ifdef BIG_ENDIAN
struct mystruct{
   unsigned short x : 4;
   unsigned short y : 5;
   unsigned short z : 7;
} test;
...

There are tons of other gotchas as well, such as using a custom byte swap macro without checking endianness, an incompatible type is used with a network macro such as ‘ntohl’, compiler-dependent option is used, macro describing a builtin numeric type is used, and many more.

Using static analysis tools not only detects porting issues, but provides metrics and reporting capabilities that allow you to prioritize and identify the scope of work involved.  For example, Top 10 porting issues (see the diagram above) helps you identify the bulk of the issues that need to be addressed.  Or you can use the component diagram report to identify specific components of your software code base you may have to concentrate on going forward.

Nice blog post from Walter Bright over at Dr. Dobbs on the Patterns of Bugs. He ties together bug patterns, recommended process changes, and the resulting productivity payoff from making these improvements. He recommends a bunch of process changes, including static analysis, code reviews, and coding standards, then goes on to review examples of different bug patterns. A few can be detected with static analysis (coding mistakes as written) but many are errors with the code as intended (something static analysis doesn’t check… that’s what testing is for). His main recommendation seems to be that bugs can often be pattern based, so once a bug is identified, take steps to remove that pattern from your code through process or tool changes.

Once the pattern is identified, then look for changes in process that will permanently eliminate that bug. Eliminate enough of the bug patterns, and you should enjoy a substantial increase in productivity.

His experience is that, over time this kind of systematic approach to fixing bugs makes developers better.

I’ve noticed in my decades of writing programs that I just don’t make the kinds of mistakes I used to. Apparently I’ve unconsciously evolved coping strategies to avoid them. Identifying and building such strategies into the process means everyone can benefit from that experience.

We certainly see that in the area of static analysis bug detection; lots of customers report that their developers make fewer mistakes, defect injection rates go down, and overall productivity is improved.

What comes first—the process or the tool?

Yes.

Any tool worth its salt should integrate into existing processes and tools.

What’s interesting and informative is seeing the similarities and differences in how the same tool is applied in different organizations, across continents and oceans.

The emphasis on quality unites everyone, but the level to which agile is adopted is what makes static analysis markets different.

No one knows this more than Mark Grice, Klocwork Director and Manager of the International Reseller/Partner Network, and Steve Howard, head of Partner Support in Europe.

Trying to talk to these guys at the same time is a challenge. Mark’s work takes him all over Asia, but I managed to pinpoint him in Japan; Steve spends most of his time traveling across Europe, but was at his home base in England—for a time, anyway.

Same difference

The European and Asian markets are all singing from the same song book when it comes to coding standards.

“MISRA is very appealing here in Europe, particularly on the developer desktop,” Steve says.

“There’s a strong focus on quality here,” Mark says about his Asian market. While MISRA is “somewhat appealing,” Mark says he gets asked about Embedded System development exemplar Reference Series (ESxR) quite a lot. It’s a coding standard similar to MISRA in that it’s aimed at embedded system development, but its adoption is more Japan-centric. For more information, see ESxR at Information-technology Promotion Agency, Japan (IPA).

Steve explains  that the ability to extend checkers to suit the needs of specific organizations is of keen interest to customers in his bailiwick.
“Sometimes organizations want to enforce their own naming conventions or code constructions, and the extensibility tools provide a very quick and effective way to accomplish that,” he says.

Difference

The developer desktop illustrates the great agile divide.

“I’d say Europe is a little bit behind North America in its adoption of agile, but there’s still the same requirement for developer productivity and speed,” Steve says.

For that reason, he sees more opportunity to penetrate the European market with a static code analysis tool for the developer desktop. The growing interest in agile in Europe gives him an increasingly receptive audience.

Japan is a bit different on this score from the rest of Asia.

Quality is seen as the purview of testing teams, and not development. That means that there’s a huge focus on quality at the end of the development life cycle, rather than the beginning, Mark explains.

Desktop source code analysis tools are a harder sell in Japan, but that may change as agile processes trickle in.

And there you have it, a quick peek at a couple of our markets.

Well no, far from it, but there has been an interesting post on this and other discussions from the browser makers themselves.  Namely Firefox has removed the RSS feed from the toolbar in 4.0.  This has sparked much conversation to bring that back.

The main reason for this is that it simply is not used, therefore having it so prominent is unnecessary.  Furthermore all sites already provide this capability on their pages, so why include one on the browser too?  There are some interesting counters to this and I suggest you have a read.

This got me thinking as well, since the code review product in Klocwork Insight Pro already has that RSS feed icon.  So all is good right?  Well the post also brings up the fact that most don’t know what this symbol means.  Do you know?  If you do know, do you use it?

In my last post, I talked about some of the reasons why I started developing Android apps. The new Android 2.3 Gingerbread is even better than Froyo. I’m talking about a whole new Nerdy and the Droid factory.

With Gingerbread released and rolled out on the new flagship phone, the Google Nexus S, the developers’ wait is over. Some of the new features and functionalities baked within Gingerbread are:

1.    More power under the hood

My Nexus One lasts me about a day running Froyo. Android is now way smarter in managing its electrons with Gingerbread. With new interface design, smart process management and improved power management, it has increased the battery life significantly.

2.    Near Field Communication

NFC provides high-speed, short-range wireless communication capabilities and is responsible for most of the Gingerbread hype. NFC has introduced many new playgrounds for Android developers such as mobile tickets, mobile payment, identity documents, smart posters, electronic keys and more.

3.    Improved audio and graphics

Gingerbread now supports extra-large screen sizes and resolutions in addition to audio, graphic and input enhancements. This is a red carpet invitation for game developers to knock themselves out!

4.    Session Initiation Protocol VOIP telephony

SIP VOIP allows enhanced video and audio calling. It also supports video conferencing, streaming multimedia distribution, instant messaging, file transfer and online games.

5.    Native sensor support

Gingerbread now natively supports sensors such as the gyroscopes and barometers.

And let’s not forget new features like concurrent garbage collection, multi-touch software keyboard, enhanced support for native code, multi-camera support and new download management.

With all these new building blocks, I can’t wait to see what new apps are going to show up on my App Market!

Happy coding!