Compiler configuration is a problem with static analysis tools.  In the past, a static analysis (or source code analysis) tool simply worked by pointing it at the source code and hitting “go”.  Now it is very different.  Without a complete understanding of the software build, including the compiler specifics, you will get inaccurate results.

Under the covers, do you really know what is happening with your compiler?  Not usually.  You make changes to your code, call your compiler or build command to compile your code, and then fix the issues.  Rinse and repeat.

But what is really important for static analysis tools, is that the compiler contains some crucial information to successfully compile your code.  Namely, the internal compiler includes and defines.  Static analysis tools must generate this data, otherwise they won’t know where the system includes and defines are coming from for your specific compiler.  Hence, the static analysis results are about as accurate as the weather man’s weekly prediction.

Luckily most compilers have a way to capture this.  For example to find out the defines and includes from gcc:

gcc -E -dM dummy.c

gcc -E -Wp,-v dummy.c

Where dummy.c is just an empty file.  This will give you a dump of all the defines and includes, respectively.  Now, when static analysis tools build their data they have a mapping of the proper defines and includes for your specific compiler and everyone is happy.

In the past, it seemed like a good idea to make compiler configuration extensible.  This meant that static analysis tools could support any compiler if you didn’t mind taking the time to build that support.  It wasn’t usually very complex but it could be prone to errors.  Instead, it makes more sense to just provide the support right out of the box, so taking the words from the late Steve Jobs: “it just works“.  As long as static analysis tools have an extensible interface, these tools should be able to support new and obscure compilers very quickly.  Make sure your static analysis vendor has support for your specific compiler that you use, and if they don’t they better turn that around in a snap.

We have blogged before about software security guidelines, but there is one we haven’t discussed.  Several years ago Microsoft published the “Security Development Lifecycle (SDL) Banned Function Calls” list.  These banned functions can be a good way to remove a significant number of potential code vulnerabilities from C and C++ code.  They provide recommendations on better or safer functions to use with the caveat that even these “safer” function should be used with care.

You can use the banned.h file to identify and obtain deprecation warnings or, even better, use this as part of your source code analysis.  Leveraging these warning as part of your source code analysis solution means you have better ways to filter and manage the solution as opposed to a dump of potentially thousands of warnings.  Add that into your code review tool and you have some good discussion points for your peer code reviews.

Like any security guideline, the question becomes how useful are these?  There is no question that these banned functions are debatable.  The complaint that I hear the most is that “n” functions can be used safely so they should not be part of the list.  But you can still get yourself in a whole heap of trouble with these functions as well.  Take this example from Micheal Howard’s blog:  Buffer Overflow in Apache 1.3.xx fixed on Bugtraq – the evils of strncpy and strncat!.

I believe there is merit in identifying these functions so you can ask yourself if you’re using them securely.  For more information and training on the Microsoft SDL you can look at the course “Intro to the Microsoft Security Development Lifecycle” on our web page.

Is anyone out there using the Microsoft banned function list religiously?

Recently I was at our European partner advisory board.  This is a session where we all get together and talk about the current market, the upcoming release and anything else to help our partners be more successful.  The most valuable sessions for myself is hearing from the partners on what works and what doesn’t.  This ranges from commercial issues to technical issues with the product.

One very clear message from all the partners was that our MISRA support was a huge plus.  Here in North America we have seen small pockets of adoption, but in Europe and even Asia it used quite a bit.  As we have mentioned in the past, it is not only automotive organization but all levels of business.

Soon MISRA C 2011 will be released.  Look forward to seeing all the changes especially with the added support of C99.

I read an interesting post on electronic imports that could contain security threats.  I can only speak from the software perspective, but I can say that most customers I’ve dealt with usually integrate some sort of software security audit process with any 3^rd-party integrator and from my experience that means adopting static analysis.  How many organizations are there that haven’t jumped on board with static analysis?  Probably more than I can count.

It would be very interesting to hear of some of the Armed Services and Intelligence cyber threats that the government has not publically disclosed.  That might be an eye opener.

The U.S. Department of Homeland Security, in conjunction with the SANS Institute and Mitre have been hard at work again.  See the article.  There are two new programs called the Common Weakness Risk Analysis Framework (CWRAF) and the Common Weakness Scoring System (CWSS).  Using these two in conjunction will help users identify the most important weaknesses for their business.  It will be interesting to see adoption in the upcoming weeks.

In addition to CWRAF and CWSS the 2011 CWE/SANS Top 25 list has been updated.  There has been a number of position changes and a few that have been knocked out and replaced by CWE-250, CWE-676, CWE-134, and CWE-759.  Not too many surprises but I never really noticed CWE-134 not in the list before.  That certainly makes sense.  However it does shock me that CWE-129 (Improper Validation of Array Index) didn’t make the list this year.  Certainly a problem that I’ve seen a ton, although it was close (#27).  To see Klocwork’s coverage of 2011 CWE/SANS Top 25 go here.

I’m sure this topic has been discussed a million times, but hey, here we go again.  A recent question came up on whether people liked their experience of Eclipse vs. Visual Studio.  Of course this brought up the advantages of one versus the other.  But is that really a fair comparison? It really depends.  What type of application are you building — a native Windows application?  Surely going with Visual Studio makes sense. But if the goal is cross-platform, then you might look at Eclipse.

Glad to see people are thinking about IDEs, but what really intrigues me about this conversation of one IDE versus another is that someone always has to add their two cents about the ancient text editors of the world.   Something like “real programmers use vi”.  Hold the phone.  Are we talking about the same text editor that requires you to memorize a gazillion key bindings?

I don’t get this.  I understand legacy use, as vi was the only available built-in text editor at the time and still is the only choice of hackers today.  But times have changed.  Anyone I’ve talked to who is using vi (or other known text editors like emacs) always seems very proud of it.  Maybe knowing how to use such a complex tool provides some self-worth.  I just don’t know.  Seems like it would be the same as me bragging about my portable Walkman or the 8-track player in my car.

Don’t the features of Visual Studio or Eclipse make you faster?  With a click of a button you can refactor your code.  With simple auto-completion the IDE eliminates simple typing (or even mistakes).  Plus with built-in tools for static analysis, test generation, etc., what are you waiting for?

So you vi/vim/emacs coders out there — tell me why on earth you are sticking with it. What makes you a better programmer using vi/vim or emacs?

A while ago I talked about memory overflows.  Now in this latest instalment, as we look at more interesting bugs, I’ve come across a new example.  Here is a situation described by a customer as “stack smashing”, which occurs when you copy a string of unknown length into a fixed buffer size.

#include <stdio.h>;

void foo(char * v) {
    char buffer[10];
    if(v != NULL)
   {
            memcpy(buffer, v,strlen(v));
    }
}

int main (int argc, char ** argv)
{
    foo(argv[1]);
    foo("the longest string you can find");
    return 0;
}

Just like the memory overflow post this is another form of a buffer overflow.  So there you have it, just more terminology to describe bad things in your code.  Gwyn promises to give a follow up to these posts with some details on how this general area (stack corruption, tainted sources, etc.) can be exploited.  Can’t wait to see that.

Will source code analysis (SCA) or static analysis change developer culture? The answer really depends on the developer’s skill set. In my experience, there are developers who are excellent at what they do (visionaries), and then there are some that just don’t get it (fence posts).

I’m not here to talk about the visionaries — they already get it. They know that SCA techniques help find critical issues early in the development cycle. Sometimes SCA finds great stuff, sometimes it doesn’t. But it’s always worth the time, because it makes developers better at what they do. In fact, it’s the visionaries who demand SCA from the outset.

Nor am I here to talk about the fence posts; they won’t last long at what they’re doing anyway.

I’m mostly concerned with the majority, the ones that fall in between. Do they find value in SCA? Will it change the way they develop code?  These developers are quite valued and extremely important to the organization — so much so that they have a full workload and tons of commitments. It’s because of this workload that you typically won’t see a shift in their work culture. When they’re asked to run SCA as part of their development process, they’ll probably accept this and give their honest opinion. They will wait to see whether it finds the so-called “silver bullet” to decide whether they are getting value out of it. In other words, if SCA finds a super-juicy bug that would have been catastrophic, it’s a winner. If not, then you’re probably not going to convince them that SCA is right for them. On top of that, they have a pretty big workload, and SCA is throwing false positives, making them spend more time.

The reality is that with SCA you may never find the silver bullet. The reality is that SCA tools throw false positives. The reality is that it will take some time up front to be proficient with the tools. Yes, these realities suck. But don’t lose focus on the prize. SCA will always pay off in the end. The value of SCA is quite clear these days (see here, here and here).

Okay, so there’s some value despite the realities, but there’s also a hidden value: training. Even though SCA may show you a defect you don’t care about, it gives you food for thought. Coding best practices are a good example. A memory-constrained shop will tell you they always need to check for NULL with memory allocation:

void *blah = malloc(10);
if (blah != NULL){
   /* Do something here with blah */
}else{
/* Do something here if it fails */
}

Surprisingly, others will tell you that the likelihood of running out of memory is very low. So why place the unnecessary checks? This and many other arguments will go on forever. But these type of questions make you think twice: should I be checking for NULL, or should I rewrite my code? So in the end, SCA gets you thinking.

A few years back a customer said they had all kinds of trouble with bugs corrupting their stack.  Even though they asked if source code analysis tools could help find stack corruption, once we got an example, we found that they were really looking for was memory overflows.  So what on earth is a memory overflow?  Does that even exist?

Yes, except it is probably not what you’re thinking, it’s not the same as a memory leak;  a memory overflow is quite different.  A memory overflow is really just a form of a buffer overflow.  The impact of memory overflow is unexpected behavior or program failure.   Take this example:

#include <stdio.h>
typedef struct s1_ {
   int i;
   int j;
   char arr[10];
}s1;

typedef struct s2_ {
   char b[20];
   char c[40];
}s2;

main()
{
   s1 block1;
   memset(&block1, 0, sizeof(s2));
   block1.i =1;
}

Here we have the case of incorrectly using ‘memset’ at line 16 where ‘sizeof(s2)’ is bigger than ‘block1′.  In fact, going back to this customer revealed that the issue was due to memset clearing much more area than intended.  If you’re using static analysis or source code analysis tools then you are probably covered by this.  You will find this type of issue usually in the “buffer overflow” category.

So, are you free of your memory overflows?

There was a recent article from Mark Pitchford titled: “Think static analysis cures all ills? Think again.” Obviously being biased working here at Klocwork, I take a major exception to what Mark has to say.

This article makes ridiculous claims. About the only thing Mark got right was that static analysis has been around for a long time. However it’s ludicrous to think that they’re the same as they were in the past. That’s like saying computers from decades ago are the same as today. The advancement has been huge for static analysis tools, especially in the last couple of years.

The author is really selling the merits of dynamic testing, which is great. Everyone should have the proper testing procedures in place. But static analysis is complementary–it’s another tool that will help you go through ALL the paths of your code to help you find bugs you’d otherwise miss.

One of the biggest reasons static analysis tools have taken off, in my opinion, is the level of integration. It’s quite simple to get results with static analysis tools with little effort. Especially compared with dynamic analysis tools.

Paul Anderson, a fellow competitor, sums it up very nicely in the comments. Check it out.