It happened again.  For what seems like the 100th time, someone reports to me that they are seeing a number of false positive reports on the resource leak checker.  For those not familiar with a resource leak, take a look at a previous post.  Although resource leaks apply across most languages, the place where this question keeps coming  up seems to always be in Java or C# code.  My last query came from Java code, so we will use that as an example.  Here was a report where the FileInputSteam is not closed on exit.

FileChannel sch = new FileInputStream(...);
FileChannel dch = new FileOutputStream(...);
try{
   ...
} catch (...) {
   ...
} finally {
   try {
      sch.close();
   }
}
...

Do you see this issue?  Clearly, they close the FileInputStream in the finally block.  The problem is that you can still throw an exception when you try to create a FileOutputStream.  The simple fix is to encompass the FileInputStream and FileOutputStream in the try/catch blocks.

Static analysis tools are great for finding bugs like this resource leak.  In this case, the report that a FileInputStream is not closed on exit wasn’t enough information to debug this.  This brings us to a very important discussion with static analysis tools–trace.

Gone are the days where a static analysis tool reports a bug at line X with no context.  Today you get detailed traces.  The trace contains conditions and assignments to give you clues about why the tool is identifying a bug.  In this example below, identifying the “source” and “sink” of any defect will help you trace through the start/end of the control flow.

More importantly in the case of Java, here we have a clear understanding where exceptions are thrown.  This would have helped tremendously for the example above.

Detailed traces are very important and can do many things for individual checkers.  So whatever you do, don’t just look at the bug description, pay attention to the trace!

If you’ve ever gone through the process of porting an application, you know the pain.  Porting can be difficult if you’re not vigilant from the outset.  There are tons of written guidelines and best practices for specific platforms or architectures, such as those going to 64 bit for Windows apps or Intel architecture and Mac OS.

In the past, we have talked about Endian issues, which are very specific to porting from different architectures (big-endian vs little-endian).  This time I want to take you through some general porting issues to show you how you can now use static analysis tools to help you through this process.

A very common gotcha are situations in which an expression is cast to a type of potentially different size.  Code written for a particular architecture’s data model can face significant challenges in porting when the new architecture imposes a new data model. For example, when porting between 32 bit and 64 bit data models, the new data model typically leaves ‘int’ at 32 bits, but uses ‘long’ as a 64 bit value, breaking any assumed equality in data type size that worked under the 32 bit data model.  Using static analysis tools can help you find this automatically.  In the example below all casts will throw an error:

void bad(){
 unsigned char uc = 0;
 unsigned short us = 0;
 unsigned int ui = 0;
 unsigned long long ull = 0;
 long l=0;

 l = (long)ull; //porting issue
 uc = (unsigned char)l; //porting issue
 us = (unsigned short)l; //porting issue
 ui = (unsigned int)l; //porting issue
}

Another gotcha is using bit fields within a structure.  When you define a data structure using bit fields, you are relying on the endian representation of a particular compiler. Moving to a different compiler can modify this layout, resulting in problems when that data structure is projected onto a byte buffer.  Again static analysis tools can be used to identify where the bit field definition exists and NOT properly covered by a big-endian or little-endian macro.  For example flag this:

struct mystruct{
   unsigned short x : 4;
   unsigned short y : 5;
   unsigned short z : 7;
} test;

But don’t flag the same thing with conditional compilation:

#ifdef BIG_ENDIAN
struct mystruct{
   unsigned short x : 4;
   unsigned short y : 5;
   unsigned short z : 7;
} test;
...

There are tons of other gotchas as well, such as using a custom byte swap macro without checking endianness, an incompatible type is used with a network macro such as ‘ntohl’, compiler-dependent option is used, macro describing a builtin numeric type is used, and many more.

Using static analysis tools not only detects porting issues, but provides metrics and reporting capabilities that allow you to prioritize and identify the scope of work involved.  For example, Top 10 porting issues (see the diagram above) helps you identify the bulk of the issues that need to be addressed.  Or you can use the component diagram report to identify specific components of your software code base you may have to concentrate on going forward.

Well no, far from it, but there has been an interesting post on this and other discussions from the browser makers themselves.  Namely Firefox has removed the RSS feed from the toolbar in 4.0.  This has sparked much conversation to bring that back.

The main reason for this is that it simply is not used, therefore having it so prominent is unnecessary.  Furthermore all sites already provide this capability on their pages, so why include one on the browser too?  There are some interesting counters to this and I suggest you have a read.

This got me thinking as well, since the code review product in Klocwork Insight Pro already has that RSS feed icon.  So all is good right?  Well the post also brings up the fact that most don’t know what this symbol means.  Do you know?  If you do know, do you use it?

I don’t know why, but every time I go back to some simple shell scripting I can never remember the one liner for loop.  For those who know what I’m talking about, it is the one command you need over and over again when it comes to performing the same thing on a set of files.  I can never seem to remember the simple syntax and it always leaves me scratching my head.

So I thought I would ask a forum on shell scripting and  was promptly answered with:

for file in `ls`; do rm -rf $file; done

I don’t think they were very friendly.  Although they have the syntax correct, please don’t try this at home.  If you don’t know why, please turn off your computer.  In the end, a simple Google search brought me to this blog with all the details.

So in the end a simple command to create copies for all my .java files to .java.bak:

for i in `find . -name "*.java"`; do cp $i $i.bak; done

Don’t forget, those are executable quotes that you’re using for the find command.  Maybe there’s a better way to do this, but in the end I find the one liner for loop very handy…if only I can remember it.

On another note, I tried this with my Windows Cygwin installation and it doesn’t work.  Anyone with insights into that?

As we have posted in the past, refactoring is a very useful technology to help developers become more productive.  I wanted to take a deeper look at how certain refactorings such as “Extract Function/Method” and “Introduce Variable” can be further enhanced with clone detection.

For the focus of this post I will concentrate just on Extract Function/Method.  Say I create some code that I know I will use frequently.  It would make more sense to create a reusable function/method.  Of course I can add a function to my file then pass the proper parameters and function call, or I can click the extract function refactoring which does this automatically for me.

void dosomething (int a, int b)
{
...
   int c = a + b;
   printf("result= %i ",c);
...
}

Now if I select the two lines and run extract function, it will ask me for a name of the function, I’ll call it “add” and we end up with this:

void add (int a, int b)
{
   int c = a + b;
   printf("result= %i ",c);
}

void dosomething (int a, int b)
{
   add(a, b);
}

Same idea for Introduce variable.  Except it works with an expression and replaces it with a variable name of your choice.

So obviously extract function and introduce variable has value,  anything you can do to make the development process more automated can only make you more productive.  But let’s take a look at how we can improve on this and show where you can really take advantage of the power of refactoring.  In most cases you are working with existing code with plenty of duplication.  This is where clone detection comes into place.  Essentially clone detection is the process of finding the same pattern you just highlighted with special attention to code semantics.  This means you now get immediate notification that there are other areas of duplication, even if variable names might be different, and replace with a click of a button.

Taking the simple extract method function, let’s say we have similar code many lines later.

void dosomething(int a, int b)
{
   ...
   int c = a + b;
   printf("result= %i ",c);
   ...
   int d = a + b;
   printf("result= %i ",d);
   ...
}

Now if we run extract method, you get notified about each clone and the choice to replace.

void add(int a, int b)
{
   int c = a + b;
   printf("result= %i ",c);
}

void dosomething(int a, int b)
{
   ...
   add(a, b);
   ...
   add(a, b);
   ...
}

This is a file level refactoring so clone detection can even apply to multiple functions in the same file!   So hopefully with this you will have a new found love for what refactoring can do you.

Today I wanted to talk about new kinds of benefits you can get from Refactoring.  Everyone knows that refactoring is the process of simplifying and clarifying code without changing the program’s behavior.  The benefits include making the developer more productive by providing tools to automatically clean up the code.  Some of you may be aware of the common refactoring such as “Rename” to rename a variable, parameter or function in your code. Or Extract Function to create a function call and body based on some selected code.

These are great and provide important value but imagine now being able to get additional benefits of reduced compile times and reduced system size in addition to the increased maintainability and reduction of complexity.  All this with a “Optimize Headers” refactoring.

For those who write Java code know of a similar refactoring called “Optimize imports”.  The optimize headers refactoring is for C/C++ code to optimize your include directives.  Specifically it is looking for extra includes and missing transitive issues.  For extra includes you need to analyze the system build structure for that file and identify if any identifiers from the included file are used.  If not then simply remove.  That will reduce your build time, system size and clean up your include structure.

Missing transitive include issue is something that consists of 3 or more files.  For example file A includes B, then B includes C.  But if file A is not using anything from B then you should remove that include and replace it with C (because file A uses identifiers from C).

I’ve added a video that shows how this is done with Eclipse.  So if you thought refactoring is just about code reuse and maintainability think again.

I recently read a book by Peter Ritchie called “Refactoring with Microsoft Visual Studio 2010” and thought I would give my review.

Great book to really help you get started with Refactoring.  Ritchie first goes into an introduction of refactoring and some of the tools available in Visual Studio 2010.  He then provides techniques to help you identify code that might need to be refactored along with examples and step by step procedures to refactor the code.

The focus of this book is not necessarily with Visual Studio 2010, as many of the refactoring examples he goes through do not even have a solution with Visual Studio, but more with a focus on C# code.  When I first ordered this book it wasn’t entirely clear to me that C# was the only language of focus.  That was a little bit disappointing for me because at Klocwork our focus is on refactoring for C/C++.  I guess I’m not surprised as not too many tools are providing a refactoring solution for C/C++ developers.  But with that said even though all examples are C#, what it really helps you understand is where and when to refactor and that is language independent.

I think Ritchie really describes the question of “why refactor” brilliantly.  The point of refactoring vs. rewriting is to fix something that’s not broken.  But as Ritchie states, he is a believer in if it ain’t broken, don’t fix it, but unless your code base defies all reality, it is likely that your code is constantly evolving and getting more complex.  Hence the need to refactor.

Another important point made by Ritchie is that refactoring falls into the developer’s hands.  It is not a task that keeps building up until you identify numerous backlog’s of refactoring.  It is something you need to do daily, as needed.

Overall I enjoyed the detail and examples provided and recommend it to any.  I hope to talk a little more on how Ritchie relates metrics to refactoring in another post.

Endianness refers to the ordering of bytes into memory. As many of you are aware, computers do ordering differently. You can have the representation of Big-endian or Little-endian.

Essentially Big-endian stores data big-end first, meaning the first byte is the biggest and Little-endian stores data little-end first, meaning the first byte is smallest.

Because all machines are different and write data either as big or little-endian, a computer could read this data incorrectly.  If you are not prepared ahead of time for heterogeneous processor architectures, then you might be in for a world of hurt. This article describes the problem in detail and also provides solutions when you’re starting off.

But what if you’re not prepared?  This can be a costly and complicated problem when developing on large systems with multiple processors or during a porting effort.  You need to be able to verify the developers provide data interaction with the target processor in the proper endian format.  Here’s a good post that talks about this issue in more details, and references a bunch of good third party articles, some of which outline how source code analysis can help; the basic idea being that source code analysis can flag instances where data is being transmitted to or from the target without being transformed.

We’ve just introduced some capabilities in this area which are described in this white paper authored by our CTO Gwyn Fisher. Check it out and let us know what you think.

I’ve talked about different metrics that are used to measure quality and the metrics that developers would use in practice.  But what about the tools themselves?  How are you measuring the value you are getting out of these tools?

In terms of static analysis, one obvious measurement is simply the bug fixes you have made.  Most organizations have a number they use to define the cost savings for each bug.  Using some research data from IBM puts the cost of fixing a bug before a release at 40-50 times cheaper.   Fixing a bug after release is the extreme and hopefully you’re finding these issues earlier and that’s where static analysis comes in.

I would think any static analysis tool offering would give you that simple graph.  Below is a couple of examples where you can track the fixes per component or per owner.

Bug fixes per compoent

Bug fixes per owner

One other thing that I think is very important is where you’re saving money.  When you run static analysis for example you have a couple of choices where you can find defects.  After the integration build or right at the developers desktop before they check-in their code.  Obviously if the developers are finding and fixing their issues while they code and before they even get checked in then it is saving even more money for the company.  Good static analysis tools will help you see the value of using it at the desktop.  A simple counter is all you need to see how many bugs were fixed in the code prior to check-in.  Immediate ROI from using static analysis.

As the topic suggests, this kind of argument has been around for some time.  Most developers can recognize the need for tools but once you start breaking the developer’s day-to -day workflow you might as well flush that tool down the drain.

What developers need is a tool that seamlessly integrates with their development environment and their workflow, so they can meet their quality goals without taking a big productivity hit.

It’s one thing to provide plug-in tools for the more popular IDEs like Visual Studio and Eclipse, but it’s an added bonus when defect detection is a seamless part of the edit cycle. No buttons to click, just continuous analysis and issue highlighting while you work.

Let’s take the analogy to the spell checker.  Initially, you had to click a button to spell check your document. That has obviously changed dramatically. Now we see any mistakes we make as we type them (and can even fix them automatically).

That’s what we were thinking when we introduced continuous analysis in our plug-in tools and our source viewer for command-line tools, Klocwork Desktop.

Here’s the spell checker equivalent for source code analysis:

The above screenshot is from our Visual Studio plug-in.

When you open or save a file, the analysis runs in the background. A bug marker in the left gutter and a squiggly line, in the true spirit of the spell checker, clearly marks the detected issue.

Find ‘em and fix’em while you work.