In my previous post, It’s time to create a custom checker…, we looked at the considerations involved in deciding which checker to create–AST or path?

In this post, we’re going to use a custom checker to enforce an internal coding standard that extends the default set of checkers in our source code analysis tool.

To do this, I’ve called upon Steve Howard, our head of Partner Support in Europe, to get us started with an AST checker to accomplish our goal.

Steve has coached many customers through the checker creation process. In his experience, the appeal of custom checkers lies in their ability to enforce naming conventions and code constructions across organizations.

The standard we want to enforce is the use of a compound statement block rather than single statements as the body of a for loop. An AST checker is the way to go because detection depends solely on the syntax of the code itself and not runtime behavior.

See the example below:

To flag this violation, we need to instruct the checker to find all instances of for loop nodes that contain a Statement node as an immediate descendant.

A tool that shows you a visual representation of the AST for the test case is quite helpful in the checker creation process. Here at Klocwork, we use Checker Studio to:

• browse the AST structure of test cases,
• identify nodes of interest, and
• test XPath-like expressions that identify node types, qualifiers, conditions and variables to traverse the AST and flag the defect.

Note: If we wanted to enforce the compound statement rule in all loops, then we’d need to have one pattern (created using the XPath-like expression) for each possible kind, such as while loops and do while loops.

Armed with the test case, Checker Studio, and a syntax guide, Steve identified the following expression that flags the infraction:

// ForStmt [not (Stmt::CompoundStmt)]

Here’s how the test case and expression appear in Checker Studio:

Golden rules

Based on his experience, Steve has a number of golden rules that get you from idea to defect detection faster:

• Start simple: Use a simple test case that contains the defect you want to detect and work with one simple pattern at a time. Add more complexity as you go along
• Start rough and refine later: Don’t worry about false positives at first. In some cases it may even be easier to search for  instances that are OK and then negate the rule at the end
• Divide and conquer: With a more complex checker, work separately on each aspect of the defect you want to detect and then bring it all together at the end for testing in Checker Studio
• Watch your levels: Make the highlighting as relevant as possible for the issue you’re trying to find. For example, “// ClassType [MemberDecls[*]::MemberDecl]” will highlight classes that match, whereas “// ClassType/MemberDecls[*]::MemberDecl”  will highlight class members that match. The rule is the same, but the focus is different
• Weed out false negatives: Add negative examples (good code) to check for false negatives

For more information about our custom AST checkers, watch our Checker Studio video.

Our marketing people here at Klocwork like to see me racking up frequent flyer miles and expending CO2 at roadshows, conferences and tradeshows. Whenever I’m out speaking, I always like to gauge audience familiarity with Static Code Analysis.

I’m happy to say that SCA knowledge has definitely increased over the years, but it is still not up to levels enjoyed by unit testing or integration testing.

What I plan to do over the next three weeks is to provide you with a history lesson on how Static Code Analysis has evolved over the past few decades (yes, it has been around that long!). The three different eras I will be addressing are The Early Years, The Early 21^st Century, and  The Present Day.

The Early Years

As I mentioned earlier, Static Code Analysis has actually been around since the time of bell bottoms, disco music, and Space Invaders (check out the Space Invaders link)–yes, the good ole 1970s. Who out there has heard of Lint (and no, I’m not talking about the fluff coming from your old bell bottoms pockets)?

Lint was one of these first-generation SCA tools introduced in the late 70s. These tools targeted low hanging fruit in C code, such as missing or extra semi-colons, missing curlicues, potentially dangerous implicit casts, and so on.

These tools were closely integrated with the compile and link process, and so this seemed like the best time to show its errors and warnings, while the developer was actually “in process” and fixing problems found by the compiler. Since these tools delivered its warnings at compile time, it quickly became a tool that was adopted and owned by the developers themselves.

Life was good. Well, until the bugs that were being found were deemed to be relatively trivial or completely erroneous (the dreaded false positive). The problem was that these tools were only able to see one file at a time, but for accurate static analysis there is a strong need to know everything that’s going on within the entire stream.

Without that vision of the entire stream, no matter how sophisticated the analysis tools are, they will make incorrect assumptions most of the time. Because of these inaccuracies, these first-generation tools never gained the widespread acceptance of software developers.

Next up will be a look at static analysis tools during a time when “Whassssuuuupp” became a household term.

Klocwork developer Russ Sherk sporting his mo'stache. It's hair today, gone tomorrow.

Time’s a precious resource, so the saying goes. Don’t waste it. That’s particularly true for developers, who live in the critical path lane.

And if there’s someone who knows a lot about time management, it’s Russ Sherk, an intermediate developer here at Klocwork, and the father of three young ‘uns. Russ works on our Klocwork Review and Klocwork Inspect products and handles licensing.

For Russ, these are lessons learned over his six-year tenure at Klocwork.

“These are things you need to think about or you won’t progress as a developer,” he says.

Here’s what to do if you want to waste time:

1. Code without a plan

It’s easy to get carried away with ideas about cool features, Russ says. This is how you burn through the days–by doing a lot of things that don’t need to be done.

The fix: Classic agile philosophy. “You have a story. Then break it up into a list of features that must be done. You take those features and you break them down into tasks taking less than a day, Russ explains. A task taking two to four hours is optimal. The team here uses XPlanner for project/story/task management. You can always augment with pieces of paper.”

2. Switch tasks constantly

Mental switching eats up a lot of time because you need to evaluate where you were before you switched gears, and then go through it again when you switch back.

The fix: Get to a completion point before switching (if you can).

3. Take the idealistic approach to bad or “delicate” code

Inheriting problem code makes debugging and feature work more difficult and error prone, Russ says.

“It’s tempting to want to fix it all–but that will always set you back.”

The fix: Fix what you touch to the best of your ability. Schedule larger-scale refactoring and reorganization work separately.

4. Build infrequently

This strategy goes beyond the individual developer.  There was a time here when there were only weekly builds (Egad! Say it ain’t so!). Even 15 minutes per build slows you down, Russ says. The push by development managers for frequent builds has paid dividends.

“Now, I can go through a build process in 2-4 minutes,” Russ says. That translates into 10 to 15 builds a day.

The fix: Frequent build cycles allow you to make your changes, verify them, and handle bugs quickly.

5. Optimize early and over-engineer

This is the dark side of planning. Over-planning and perfectionism can be paralyzing.

The better approach is to keep it simple, he says.

The fix: Plan the bare minimum to get your feature working. After the feature is in place, it’s time to optimize and refactor.

And there you have it. Surprisingly, World of Warcraft didn’t even make the list.

!(social media)

I will probably get flack for this but I am going to exclude web developers from this discussion of adoption rates about social media in the developer sphere.

Having moved through the technical streams over to the dark side of marketing, I have learned to challenge assumptions and here is one of mine I think needs testing.   In this new age of “social media” and interaction, I have yet to see the leadership in the developer community make any substantive use of it.   I would love to be proved wrong on this one.   Social media, in my view, is really just branding what people have been doing for years: using peers to converse and exchange information on topics and facilitating interaction, even for niche subjects like the merits of static code analysis in mission critical applications.

The adoption rate of the “formal” social media is what I am interested in.  The blogs, twitters, facebook, digg,  etc, you know the brands that I mean.  I have been looking for weeks to find any concrete data on adoption rate and have been hard pressed to find much.

• Technorati  (March 2008- State of the Blogosphere) – 26.4 million blogs vs less than a handful about software development.
• Google Trends indicates that the ratio of software blogging to the main stream is 258 times less.
• Digg has just over 18583 diggs for software development versus over 3 million for marketing
• Twitter volumes are similar 13 900 versus 1.4 million

Why hasn’t the paradigm shift happen here like in other industries?  Online marketers are eating up social networking on Himalayan scale, so why not in development circles.  Speculating on human behaviour is not without its caveats but are technical people so different from say marketers or bus dev people?  In a nut shell – yes.

It’s  not to say developers aren’t social, in many ways the development community has been the leading the wave [Yes, that is an intentional pun for the upcoming Google Wave]. I would argue that software development has been social for well over a decade as best exemplified by the open source movement.  Some of the greatest advancements in software design and productivity have come from major collaborative efforts such as the  LAMP stack, OpenOffice and Android just to name a tiny few and open source has lead to the rise and fall or changed of direction in many a company – see Apple adopting the Linux kernel etc.

My conclusion on all this: the software development community has voted with their feet.  They do not need yet another vehicle to find their voice when they already use mechanisms (open source collaboration, forums, community websites etc) that do the job quite nicely thank you very much.

So if you disagree, take up the sword and prove otherwise.

PS. And yes I get the irony of writing a social media piece about software development on a blog. :-)

Just returned from ESC San Jose, where I spent a very enjoyable few days surrounded by the “real men” of the programming world. Forget your managed language environments, forget abstractions or object oriented fantasies of design, forget processes like Agile, these guys spend their days down at board level working in assembler and occasionally sticking their heads up into the rarified world of C (but only, you know, for stuff that doesn’t really matter…).

Hell, most of the time the hardware they’re programming is custom built just for that project, sans O/S because, you know, why would you want that crap to get in your way. One guy that stands out in the procession of awesomeness was describing his ASIC to me, asking if we could help with stack overflow problems, and launched into a necessarily abstracted description of the fact that their (classified) device didn’t really have a stack, actually, but it was kind of a well known address range that they normally treated like a stack, although not always because, you know, stuff sometimes gets in the way, so if he, like, stuck something in there that was too big, could we tell him?

God, I love these guys…

Now I began programming on boards in Z80 assembler, so trust me that I do actually know what the heck they’re doing and why, but over time I’ve followed what has felt like a fairly natural migration away from the kind of “if I want to light up that LED I have to store 0x2d in address ‘x’” programming to C, then C++, and more recently to Java and C#. Of them all, I think C++ is probably my favorite, simply because it’s low-level enough to be useful, and high-level enough to let me express myself without having to think too much about it. Frankly, in my opinion while good developers are good in any language, really good developers find their own way and then excel at it. So yes, I can probably program in any language given a few days of spin-up time, but frankly I’m too old and too cranky to get all fired up over the latest innovation of hiding-the-useful-stuff-from-me just so I can do it a bit faster.

Note that I’m not espousing any kind of intelligentsia-sponsored BS argument over the “value” of OO languages over procedural or vice versa (for more vitriol than typically fits on one page, check out this beauty from Torvalds…). And no, I don’t use STL so just shoot me, but it sucks so get over it. And if you get your rocks off over Python or Ruby or Haskell or whatever new lambda calculus-based micro language you’ve just stumbled over, have fun and get your job done, there’s enough of them for everybody, after all.

But don’t try to convert me. Proselytizing is always ugly, so just step away from the bong and let’s all be friends here. The Urban Dictionary nicely summarizes things:

The men who program in C++ are Real Men. The women who program in C++ are Real Men too.

Substitute the name of your favorite language in there in place of C++ and you’ve got the way most developers think about their own language of choice. How about you? Any favorites out there you’re willing to get boiled in the pot for?