In my previous post, It’s time to create a custom checker…, we looked at the considerations involved in deciding which checker to create–AST or path?

In this post, we’re going to use a custom checker to enforce an internal coding standard that extends the default set of checkers in our source code analysis tool.

To do this, I’ve called upon Steve Howard, our head of Partner Support in Europe, to get us started with an AST checker to accomplish our goal.

Steve has coached many customers through the checker creation process. In his experience, the appeal of custom checkers lies in their ability to enforce naming conventions and code constructions across organizations.

The standard we want to enforce is the use of a compound statement block rather than single statements as the body of a for loop. An AST checker is the way to go because detection depends solely on the syntax of the code itself and not runtime behavior.

See the example below:

To flag this violation, we need to instruct the checker to find all instances of for loop nodes that contain a Statement node as an immediate descendant.

A tool that shows you a visual representation of the AST for the test case is quite helpful in the checker creation process. Here at Klocwork, we use Checker Studio to:

• browse the AST structure of test cases,
• identify nodes of interest, and
• test XPath-like expressions that identify node types, qualifiers, conditions and variables to traverse the AST and flag the defect.

Note: If we wanted to enforce the compound statement rule in all loops, then we’d need to have one pattern (created using the XPath-like expression) for each possible kind, such as while loops and do while loops.

Armed with the test case, Checker Studio, and a syntax guide, Steve identified the following expression that flags the infraction:

// ForStmt [not (Stmt::CompoundStmt)]

Here’s how the test case and expression appear in Checker Studio:

Golden rules

Based on his experience, Steve has a number of golden rules that get you from idea to defect detection faster:

• Start simple: Use a simple test case that contains the defect you want to detect and work with one simple pattern at a time. Add more complexity as you go along
• Start rough and refine later: Don’t worry about false positives at first. In some cases it may even be easier to search for  instances that are OK and then negate the rule at the end
• Divide and conquer: With a more complex checker, work separately on each aspect of the defect you want to detect and then bring it all together at the end for testing in Checker Studio
• Watch your levels: Make the highlighting as relevant as possible for the issue you’re trying to find. For example, “// ClassType [MemberDecls[*]::MemberDecl]” will highlight classes that match, whereas “// ClassType/MemberDecls[*]::MemberDecl”  will highlight class members that match. The rule is the same, but the focus is different
• Weed out false negatives: Add negative examples (good code) to check for false negatives

For more information about our custom AST checkers, watch our Checker Studio video.

So it’s that time again, when our team finally gets to sleep for a day or two and get their lives back in order. What am I talking about?

Answer: Klocwork Insight 9.5 releases today and boy are we ever happy to see it go live — we’re “out there Jerry” and yes, of course we’re loving every minute of it.

This has been a long release, taking significant research and development to bring to fruition, all the while continuing to release more traditional shipments as we went, but finally culminating in a new, game-changing technology for source code analysis. On-the-fly, as-you-type, instant-like-for-reals, call it what you like… full-on, in-depth C/C++ analysis performed as the developer enters their code, using the “squiggly line” usability metaphor created by spell checkers. It’s one of those “why would you do it any other way” moments and we’re happy to be unique.

Not to be outdone, our web tools team has done amazing stuff with a complete redesign of our Review and Inspect tools, showcasing an awesome look and feel that leverages the toys that come with HTML5 and, amongst many other new capabilities, brings drag/drop pivot report design to the web for on-the-fly metrics and trending analysis that managers and development leads will just eat up.

To our customers, our partners and our friends in the industry, we’d like to say Welcome to Insight 9.5, hope you enjoy it.

You’ve been using source code analysis on your integration build or your desktop, or (ideally) both. And then there’s “a situation”.

The situation

Either you:

• Noticed a false negative you want detected, or
• Need a way to enforce a corporate coding standard, such as the requirement for the use of  a compound statement block rather than single statements as the body of a loop.

Now what?

Time to create a custom checker, that’s what. But what kind of checker?

Source code analysis involves two families of checkers, those that involve:

• Abstract Syntax Tree (AST) validation, and
• Code path analysis.

An AST provides a tree-based structural representation of the source code. An AST checker allows you to pinpoint problematic syntax using XPath or XPath-derived grammar to define the problem you’re looking for. AST checkers (our version is called Klockwork AST checkers, or KAST for short) don’t require program execution to run; they detect defects right away on source code.

Code path analysis, on the other hand, targets defects related to value tracking at program execution time. Instead of style violations, you’d use a path checker to answer questions such as:

• Is this newly-created object released before all aliases to it are removed from scope?
• Is this data object ever range-checked before being passed to an OS function?
• Is this string checked for special characters before being submitted as an SQL query?

To create a path checker, you don’t need to know how data is tracked by the checker. What you do need to know are the function types and values you want to track for the analysis starting point and the analysis end point where the defect (or event) is recognized and reported.

Which checker when?

Create an AST checker when the problem you want to detect:

• is a local defect
• does not involve program execution
• has to do with the way the program was written
• does not involve tracking a value
• does not involve a path

Create a path checker when the problem you want to detect:

• involves tracking a value
• has a starting point (where the analysis starts) and end point (where the defect is detected)
• involves program execution

Stay tuned for the next post in this series on best practices for AST checker creation.

For more information, see Writing custom checkers with Klocwork Extensibility or check out our member discussions in the C/C++ custom checkers forum.

–With files from CTO Gwyn Fisher

As a pretty avid Apple user (2 iPhones, 1 iPad2, iMac, iPod Touch, MacBook Pro, etc.), and the fact that I work in the business of software quality and security, I must admit that this article caught my attention. The article outlines how a well-known security researcher, who focuses on Apple, has found a software flaw in the iPhone and iPad, which could allow hackers to build malicious apps.What makes this even more scary is that the Apple Store may not catch these malicious apps.

To add another twist to this story, the researcher in question has been ejected from participating in Apple’s developer programs. Read about that here.

Are we now getting to the point where hackers are going to start trying more aggressively to exploit Apple products? In a survey done in 2010, over 50% of respondents thought Windows was either “very” or “extremely” vulnerable compared to only 20% for Apple. I wonder if that has changed? More importantly, do I need to start worrying about my daughter downloading the Archie comic app from the App Store?

The question of “what’s the right iteration length” may not be as interesting as any of the questions found here (gum really doesn’t stay in you for 7 years. Who knew?), but it is a common question from organizations moving to agile development. You can certainly get a lot of different opinions on this from a Google search, but since you’re reading this now, I’ll give you mine, based on personal experience.

A number of years ago, one of the projects I was PM on decided to try out Scrum. I had attended some Product Owner training, and our soon-to-be Scrum Master had been on some training as well, but we were very green and decided to approach things with a “let’s see what works best for us” mentality. At the time, we thought the best way for us to get immersed and efficient with Scrum was lots of repetitions. We went with 1-week iterations, thinking that by having a rapid and regular cycle of sprint planning meetings, demo meetings, retrospective meetings, etc. we would learn more quickly the “proper” way of doing development with Scrum.

We certainly did learn a lot during our first 3 or 4 sprints, mainly that having this regular weekly cycle of meetings was just too much overhead, and the actual amount of value produced at the end of each sprint was too little. Next on our list, the 2-week sprint.

The 2-week sprints worked great for us, and we saw the differences from the 1-week sprints almost immediately. We were producing what we thought was a good amount of value from each sprint, but with a better and more balanced level of overhead. We hit our groove and established a good cadence with these 2-week sprints, and from the looks of the burn-down chart, we were becoming a more efficient team with every sprint.

The team definitely was cruising and enjoying the pace, but the holiday season snuck up on us and we thought that it might make sense to make some adjustments to deal with the vacation time various team members would be taking.

After collecting everyone’s vacation schedule, we were able to determine a start and finish date for our “holiday sprint” that would essentially start when everyone was still in the office, and finish when everyone returned from their vacation. Call it either luck or good management, but we had planned a 4-week sprint. I won’t go through all the gory details, but let’s just say that upon reflection, the 4-week iteration just felt wrong.

The initial planning session felt harder to estimate the amount of work we could do. The cadence we developed didn’t show itself, and it really felt like we never gained any momentum during the 4 weeks. Now I’m sure that the whole holiday season thing played a role in this, but it was our least effective iteration ever, and by a lot. We never tried the 4-week iteration again.

The bottom line is that all teams are different and need to go with the iteration length that feels right for them. For us, the 2-week one was best.

For the record, I have always wondered if the 7-year rule for chewing gum was true. Glad to hear it isn’t.

Compiler configuration is a problem with static analysis tools.  In the past, a static analysis (or source code analysis) tool simply worked by pointing it at the source code and hitting “go”.  Now it is very different.  Without a complete understanding of the software build, including the compiler specifics, you will get inaccurate results.

Under the covers, do you really know what is happening with your compiler?  Not usually.  You make changes to your code, call your compiler or build command to compile your code, and then fix the issues.  Rinse and repeat.

But what is really important for static analysis tools, is that the compiler contains some crucial information to successfully compile your code.  Namely, the internal compiler includes and defines.  Static analysis tools must generate this data, otherwise they won’t know where the system includes and defines are coming from for your specific compiler.  Hence, the static analysis results are about as accurate as the weather man’s weekly prediction.

Luckily most compilers have a way to capture this.  For example to find out the defines and includes from gcc:

gcc -E -dM dummy.c

gcc -E -Wp,-v dummy.c

Where dummy.c is just an empty file.  This will give you a dump of all the defines and includes, respectively.  Now, when static analysis tools build their data they have a mapping of the proper defines and includes for your specific compiler and everyone is happy.

In the past, it seemed like a good idea to make compiler configuration extensible.  This meant that static analysis tools could support any compiler if you didn’t mind taking the time to build that support.  It wasn’t usually very complex but it could be prone to errors.  Instead, it makes more sense to just provide the support right out of the box, so taking the words from the late Steve Jobs: “it just works“.  As long as static analysis tools have an extensible interface, these tools should be able to support new and obscure compilers very quickly.  Make sure your static analysis vendor has support for your specific compiler that you use, and if they don’t they better turn that around in a snap.

Over the past few years I’ve talked to a number of customers in the embedded software and medical devices industries, and I continue to see a significant number of these organizations either moving to, or planning on moving to agile development processes.

With all of the inherent challenges for agile in these organizations such as standards/regulatory compliance, hardware changes and integration, security issues, etc. I must say that I’m a little shocked that customers are moving away from their current processes towards something like agile. Add to this the fact that the Agile Manifesto specifically states “Working software over comprehensive documentation” and it doesn’t exactly sound like agile is a great fit for embedded systems in general, let alone medical device.

Now, don’t get me wrong, I am a huge proponent of agile, and I certainly realize that there are many pros for moving to agile, and these have been well documented, but I have to wonder just how agile are these specific industries going?  I would bet that most (all?) of these organizations are adopting some of the key fundamentals of agile, but to say they are going “all in” would be a bit of a stretch.

Even industries heavy on process (because of regulatory requirements) are taking the leap into agile. How agile are they?

Looking at the manifesto a little closer, some of the principles are fairly generic and feel more like common sense than anything revolutionary, so they probably apply to any industry. There are a few principles however that are fairly easy to imagine in these industries, such as:

• getting all stakeholders involved in defining requirements (Principle #4), or
• embracing more face-to-face conversations (Principle #6), and
• simplicity, or minimizing the amount of work not done (Principle #10).

But do people really think that Principles #1 (early and often delivery of software), and #2 (welcome changing requirements) really apply to the embedded or medical devices industries? Personally I don’t see it.

So what do you think? Are the embedded software or medical devices industries capable of going full out Agile?

We have blogged before about software security guidelines, but there is one we haven’t discussed.  Several years ago Microsoft published the “Security Development Lifecycle (SDL) Banned Function Calls” list.  These banned functions can be a good way to remove a significant number of potential code vulnerabilities from C and C++ code.  They provide recommendations on better or safer functions to use with the caveat that even these “safer” function should be used with care.

You can use the banned.h file to identify and obtain deprecation warnings or, even better, use this as part of your source code analysis.  Leveraging these warning as part of your source code analysis solution means you have better ways to filter and manage the solution as opposed to a dump of potentially thousands of warnings.  Add that into your code review tool and you have some good discussion points for your peer code reviews.

Like any security guideline, the question becomes how useful are these?  There is no question that these banned functions are debatable.  The complaint that I hear the most is that “n” functions can be used safely so they should not be part of the list.  But you can still get yourself in a whole heap of trouble with these functions as well.  Take this example from Micheal Howard’s blog:  Buffer Overflow in Apache 1.3.xx fixed on Bugtraq – the evils of strncpy and strncat!.

I believe there is merit in identifying these functions so you can ask yourself if you’re using them securely.  For more information and training on the Microsoft SDL you can look at the course “Intro to the Microsoft Security Development Lifecycle” on our web page.

Is anyone out there using the Microsoft banned function list religiously?

One of the common challenges we hear from customers regarding their software security assurance programs is developer education. Sure, there are many great tools out there that can help with security, but when it comes down to it, if you’re going to truly build a culture of secure software (and not just audit your system now and then), your development team needs to be well versed on key security concepts, defensive coding principles, common attack vectors, not to mention the ins and outs of specific coding vulnerabilities like buffer overflows.

Well, we agree. That’s why we’ve partnered with our friends at Security Innovation to make some of their developer eLearning courses available for free on the new, revamped Klocwork University. I encourage you to check out the Secure Coding for C/C++ course – it’s approx 60 minutes in length, features interactive material, and is a great introductory course into many of the key concepts required to build secure software. We also have a course on Microsoft’s Secure SDL and the OWASP Top 10. Check it out!

Klocwork Developer Network presents Klocwork University, which consolidates all our online learning resources onto a single page.

Klocwork University is your one stop for self-paced online learning and how-tos about:

• Setting up and using our static analysis tools on your desktop or integration build
• The latest trends in software security
• Agile coding practices and how they intersect with static analysis
• Klocwork product overviews

At Klocwork University you’ll see helpful descriptions of:

• In-house and partner-generated e-learning courses
• Video how-tos
• Webinars

After you browse our offerings on the Klocwork University page, click your selection and access your resource. If you’re not already logged in to the Klocwork Developer Network, you’ll be prompted to log in or register to use these free resources.

This change pulls the course content descriptions from behind the login wall to provide a searchable list for members and non-members alike.

At Klocwork University, you get the information up front and you can schedule your pub breaks when and where you want. Join today. There’s no free beer though.