In a previous post, I talked about different types of metrics and how they can apply.  These were mostly focused on a system level and provided insight into system complexity from  release to release or build after build.  Now, let’s take a look at some other metrics and how they can be used effectively with the developers.

There are literally hundreds of metrics you can measure, so what is really important?  That’s a tough question because it depends on your organization.  Perhaps you have some coding standards in place or simply want to measure some basic functionality to help developers improve code quality.

Here are some of the more common metrics I’ve seen used in practice:
•    Lines of code
•    Number of functions/methods
•    Commented code
•    Nested includes
•    Number of compiler directives

Most of the metrics listed above are about maintainability.  The more lines of code you have or the more of functions/methods, the more you have to maintain for an individual file.  If these values are high, then perhaps it deserves a new file or new function.  The opposite end is commented code, where you obviously want more code commented.  Nested includes are a great way to alert the developer that adding a new include has all of a sudden added a long compile tool chain, potentially increasing your compile time and size of your system.  Tracking compiler directives is something, I think, is really useful–especially with platform specific code.  Compiler directives are a great way to give you visibility on how much you are using conditional compilation and hopefully, gives you a good understanding of the scope if a new platform is added.

Thresholds are the way to use any of these metrics.  For example, create a threshold that would throw an error if the number of functions per file is greater than 100.  Or,  first give a warning to developers when the number of functions exceeds 75 per file and then, if it goes over 100 print an error.  But what values should you use?  You can use the standard deviation to figure this out or use coding standard guidelines.  For example, many standards say that a method should not reach over 1000 lines or code, or the maximum level of include nesting should not exceed 8.

If you find that these metrics are not enough, start combining them.  Any good tool worth its  salt will let you customize these core metrics, so you can start creating your own calculations.  One example is using the lines of code metric with the commented code metric to create a calculation on the percent of comments.  We are all guilty of not documenting our code.  Use percentage of comments to make sure everyone knows when they are under the norm,  so if you have less than 10% of your file commented you get an error and less than 20% you get a warning.

Showing developers potential areas of concern will help them to be smarter at what they do and spend more time focusing on what they are good at–innovating.

Refactoring is a vital component for software developers, helping to prevent their projects from becoming unusable, and unmaintainable spaghetti code. Equally important to some developers, is the notion of refuctoring…check out this tongue in cheek look at Refactoring vs. Refuctoring. Be sure to check out the slide deck at the end.

Refuctoring describes the process of making your code unmaintainable by anybody but yourself.  I love some of the examples of Refuctoring such as “Pig Latin”, “Treasure Hunt” and my personal favorite “Stating the Bleeding Obvious”:

For example:

//initialize a to 1
int a=1;

Not that I’d ever do that (pause while I go clean up some code). Ahem, right anyways I thought I’d throw out some other refuctorings:

1.    “Catch me if you can” – Use so many goto statements that it will make anyone’s head spin.  Especially when you start adding backward goto’s.  Take a look at the CVS source code,  they have some nice (nasty) examples.

2.    “Giant tar pit of hell” – This is hard to blame one single developer as it really encompasses many developers getting together to create one big cohesive piece of crap.  You know you have a problem when you run out of printer toner trying to print these.

"Giant tar pit of hell"

3.    “WTF” – Using names that no one will know…ever.  If you create a bankaccount object, instead of calling it ‘x’, here’s a wild idea, why not call it ‘bankaccount’.  Let’s face it we are all guilty of this.

Now if you really want to be special (I don’t mean in a good way), try combining #1 and #3:

int afunction()
{

there:
   ...
   if(something)
   {
      goto here;
   }
   if(somethingelse)
   {
      goto there;
   }
here:
   ...
   if(somethingelseagain)
   {
   goto end;
   }
end:
...
return;
}

I have to admit, I’m not winning any awards either. It is certainly time to get on the refactoring bandwagon.

How do you measure your software?  There are simple metrics that help with quality, such as keeping track of the number of bugs or security vulnerabilities in your system.  Trending these metrics is a no-brainer. When trending is in place, action can be taken because everyone knows 6 security vulnerabilities is worse than 5.  But what about other types of software metrics (and there are many)?  Have you ever heard of a maintainability metric? Halstead program volume? McCabe cyclomatic complexity?  Coupling/Cohesion?  The question becomes what do you do with these metrics and are they valuable?

Choosing a metric will really depend on what you’re after.  A good reason for measuring your code is to get predictable quality.   If you don’t have a metric in mind, the easiest place to start is with McCabe’s cyclomatic complexity metric.  I’ve seen many software organizations implement this as a good measure to help predict system “complexity”.  In other words, to help them understand where they may need to refactor or redesign their code.  McCabe cyclomatic complexity uses a measure of the linearly independent paths in the source code and is measured on functions or methods.

McCabe’s Cyclomatic complexity uses values to define what is complex.  Something greater than 20 is considered very complex.  You should think about re-writing that function because it is getting out of control.  Since the inception of McCabe’s Cyclomatic complexity metric,  several other variations have appeared, including Extended Cyclomatic Complexity and Plain Cyclomatic Complexity.  Back to the question, with so many metrics, which ones do you use and are they valuable?

No one can answer that question. In fact, software metrics is quite ambiguous.  It is hard to find anyone who says,’Thou shall use metric “x” because it will help you improve quality by “y” amount.’  The value “x” and “y” just don’t exist (although many have tried to put some data together).  Even more ambiguous are the values that may be defined with these metrics.  Don’t get caught up with these values; they are really arbitrary.  I’ve run into organizations where the majority of their code was deemed “very complex”.  Does this mean they should redesign their entire code base?  Certainly not. These numbers will vary depending on what you’re building.  So be careful if you use the “recommended” values for any metric.

Instead of focusing on the value of your next metric, what you really should be doing is trending that metric.  Find out if that value went up or down.  Up bad; down good.   Taking it one step further (if you really have a “thing” for the values), you could start by finding the standard deviation of your metrics.  In other words, find the average value of any metric, say complexity, plus the standard deviation.  Now, you can keep track of that value knowing that if you go outside your bounds of deviation, then you may want to look.

Software metrics certainly have their place and can help give some predictability on your system.  In another post, I’ll talk about how you can take some low level metrics for the developers and give them insight into the software system.

In my previous blog post, we talked about the value of compiler warnings and reasons to have source code analysis. Now, I’d like to get into the value of coding standards and touch on how you can fit this altogether.

Coding standards are a set of rules or guidelines usually created as part of an industry. The goal is simple, provide guidelines, so you can create better code and increase your code quality. Probably the most common coding standard I run into is called MISRA C. This is a standard created for C code in 1998 and revised in 2004. A new standard from MISRA was released in 2008 for C++ code. MISRA was developed for the Motor Industry but has since been adopted by many other industries. Other coding standards such as Joint Strike Fighter are focused on other industries, such as the aerospace world.  And there are more generic types of guidelines, such as the Power of 10, which contains more high level practices.

Either way, these standards cover everything from simply “thou shall comment code” to more specific coding no-no’s. So, how do you apply these to your process?

The advantage of these coding standards is that compliance is something you can quickly scan for using source code analysis. Any source code analysis tool worth its salt incorporates these standards into their issue checkers.

Implementing a solution for developers is key to this process. After developers check to ensure there are no compiler errors (or warnings!) they can run another process (or integrate into your existing process) using source code analysis techniques to find infractions with various coding standards in the code.

Remember that compiler warnings can be very helpful, so use them. Don’t be surprised when your source code analysis vendor asks if you are using your compiler warnings on your next checker feature request. Once you have cleaned up your compiler warnings and you want to take the next step to improve code quality, there are many good coding standards that will bring up the quality of your code. Use source code analysis tools to help you automate this process and you will guarantee a better report card.

We are starting to see a large amount of Android phones such as the Droid and Xperia X10 (see a review here) and the (soon-to-be-released) first Google phone, Nexus One. With this, expect the number of apps to increase significantly.

Droid vs. iPhone

So with the increased number of apps, do these developers have the right tools to find and fix bugs? Take a look at the leader of phone applications-iPhone. There have been several posts (here and here) that recommend using the Clang static analyzer. Apple has taken it one step further, apparently rejecting iPhone apps that access private APIs. But Clang won’t help you with Java apps.

So what do the Android developers have? Android is just Java, so there are lots of tools, right? Certainly there are static analysis tools, profilers, unit testing tools and many more. But are these tools really taking into account the Android specifics?

Let’s take an example of a resource leak. Resources such as streams, connections and graphic objects must be explicitly closed; otherwise, you run the risk of throwing exceptions depending on the open resource.




For example:

1 static final String propertyFile = "my_config.ini";
2
3 static String getProperyFromConfigFile(String name)throws IOException {
4    Properties prop = new Properties();
5    FileInputStream st = new FileInputStream(propertyFile);
6    prop.load(st);
7    return prop.getProperty(name);
8 }

Here, a resource leak should be identified since line 5 opens up a FileInputStream, but is never closed before exiting the method. Now, this is all well and good and valuable to be found in any Android specific code, but what happens if I’m using built-in classes from the Android SDK?

For example:

1 public boolean onKeyDown(final int keyCode, final KeyEvent event) {
2    if (keyCode == KeyEvent.KEYCODE_DPAD_CENTER) {
3          final MediaPlayer player = MediaPlayer.create(this, ringtoneUri);
4          player.start();
5    }
6    return super.onKeyDown(keyCode, event);
7 }

Here, you have a situation where a MediaPlayer resource is created at line 3, but never closed on exit. Without the knowledge that MediaPlayer is a resource that should be closed, you will miss this type of issue. This extends to many resources and different issues. You can also have Android-specific null pointer exceptions and use of free issues.

Let me know if you’re doing Android development. I want to hear what you are doing to find these kinds of bugs.

In the first blog series, we discussed the value of compiler warnings and wondered why a static analysis tool would have similar error checking features. In this installment, we want to dive deeper into this question by reviewing errors that can be found by compilers, why they matter, and what limitations compilers have in this area.

Let’s take an example of the “implicit int” rule:

int foo() {
   const x = 0;
   return x;
}

This is a situation where failure to specify a type results in this compiler warning from (gcc v.3.4.4) or Microsoft cl (v.14):

gcc -Wall -c main.c
main.c: In function `foo':
main.c:2: warning: type defaults to `int' in declaration of `x'

cl -c -Wall main.c
main.c(2) : warning C4431: missing type specifier - int assumed. Note: C no longer
supports default-int

You can’t rely on the standard C/C++ implementations to support the implicit int anymore and these compilers alert you to that.  I do have to say, I’ve never seen anyone do this in practice, but it’s nice to know it’s there.

Let’s look at another example:

void foo() {
   if (sizeof(char) < 2)  // defect - the condition is constant
   {
      /* ... */
   }
}

The issue above is that the condition is constant.  See the C99 standard for details on this (section 6.6).  If we run the cl compiler we get:

cl -c -Wall main2.c
main2.c(2) : warning C4127: conditional expression is constant

Here, the cl compiler finds the issue, gcc does not (well, at least my version).   Okay, interesting let’s take a look at a C++ example:

class A
{
   public:
   // non-virtual destructor
   ~A();
   virtual void f1();
};

With this example, if you run either gcc or cl you get the same thing:

gcc -Wall -c main3a.cpp
main3a.cpp:2: warning: `class A' has virtual functions but non-virtual destructor

cl -c -Wall main3a.cpp
main3a.cpp(7) : warning C4265: 'A' : class has virtual functions, but destructor is
not virtual instances of this class may not be destructed correctly

According to the output from both compilers, we made a boneheaded mistake and forgot to assign the destructor as virtual.  Let’s go one step further and define a new method:

void deleteA(A *a) {
   delete a;
}

This method adds a new level of complexity.  When an object of a class derived from the given one is deleted through a pointer to the given class, the destructor of the derived class is not executed, and members of the derived class are not disposed of properly.  In this case, you will not get any warnings from any compiler.  The difference here is that compilers only work within the context of the file/function.  In this case, you are out of luck with compilers, but luckily source code analysis excels in this.

So, the message here is that compiler warnings are quite useful, but they do have their limitations.  Not all compilers report the same things consistently, nor do they cover analysis beyond a single function or file.  Still, make sure you run the compiler warnings, then implement static source code analysis as part of your process to go deeper and find some more complex issues in your code.

For the next blog of this series I’ll cover coding standards and where they fit in your code quality process.

In a previous posts I reviewed the Top 5 C/C++ and Top 5 C# quality bugs that I that I see time and time again looking at customer code. I wrote my Java Top 5 with an embedded programming focus and the folks at www.embedded.com decided to publish it on their site. Here’s a snippet below and the full Top 5 Java bugs article can be found here.

While C dominates as the programming language of choice for embedded development, the use of Java is definitely on the rise. In fact, according to a recent VDC survey, 12.3% of respondents currently use Java in the embedded space, and 17.9% expect to be using Java in the next two years.

For those transitioning from embedded development using C, you might find yourself falling into the hype that Java is a “safe” language. For example, Java developers face no requirement for managing memory associated with objects. However, this is where the trap may be laid. Even though there’s no need for memory management, developers may need to keep track of specific resources the object allocates. This is especially true in an embedded context where resources are often constrained. Even for experienced developers, these traps pop up time and again and can easily jeopardize your code quality and security.

Here’s a round-up of the top five programming issues developers should be aware of in embedded Java development [More...]

In this 3 part blog series I want to cover general misconceptions with static analysis coverage.  This will include a discussion about:

• compiler warnings available,
• different types of style issues including coding standards, and
• your available options to fit them into your formal process.

Very often customers ask why we don’t cover specific checkers.  We always get great feedback on high value checkers that they would like to see.  But occasionally we get the request to find simple compiler warnings or code style issues.

For the first part of this series I want to focus on compiler warnings.  These are not the compiler errors such as syntax/parse errors you get with a compiler.  Instead, I want to concentrate on those pesky compiler warnings that still let you build your application, when you really know you shouldn’t.  We have all experienced compiler warnings that are just plain confused with your code.  But on the whole, you are guaranteed to find some pretty big blunders.

Most modern compilers focus on providing more details about compiler warnings.  These can be very valuable as it helps find many of those plain dumb mistakes.  It varies by compilers, but many find things such as constant expressions from conditionals, returning from a void function, assignment in condition (use = instead of ==), suspiciously-placed semi-colon and many, many more.

To find some of the issues, you usually need to provide a compiler flag -Wall.  For example:

    gcc -c -Wall foo.c 

Make sure you read your compiler documentation for available warnings.  Here is the gnu gcc compiler and Microsoft cl compiler docs.

Given that every compiler on the market provides its own “checkers” for compiler issues, does it really make sense for static analysis to get in there and detect these issues again?  I strongly believe that every developer should ALWAYS clean up their compiler warnings before going onto static analysis.

But you will still find static analysis tools providing these capabilities.  Why?  Well, first and foremost, not every compiler has the capability to find simple coding issues.  The other reality is that not everyone checks the compiler warnings…(we all know who we are).  Or sometimes you just want to run one tool.  In other words get the more complex bugs with static analysis along with the compiler warnings.  It is for these reasons that static analysis tools have introduced many of these low-level issues.

For the next part of this blog series, I want to go into the details of compiler warnings and some of the things that coding standards are doing as well.

Overall a good show at ESC.  Always well run and put together.  In terms of vendors we had some great discussions especially with Electric Cloud.  We managed to see a great demo of ElectricInsight.

ElectricInsight is very visual giving you an incredible view of the structure of a software build.  Not only a tool to help you visualize the impact of adding nodes to ElectricAccelerator cluster but it helps with diagnosing your build problems in seconds.  Really cool dependency analysis with a click of a button.

In the end there is no doubt the economy has taken its toll on ESC Boston 2009.  By far one of the worst turn outs for the exhibit show floor.  But one thing I can take away from this, I’m the Wii Jousting champ!

In a previous post I provided the top 5 C/C++ quality bugs that I that I see time and time again looking at customer code.  Time for the C# version:

1.    Null pointer exceptions from a method

1                  public class A {
2                      public A foo() {
3                          if (flag)
4                              return null;
5                          return new A();
6                      }
7
8                      public void var() {
9                          A a = foo();
10                         if (flag)
11                             a.foo();
12                     }
13
14                     private int flag;
15                 }

This is the most common issue I see.  In this example a warning is issued at line 11 for a possible null pointer exception.  Essentially there is a potential for a null value from the method foo().  I believe I see these issues more than any others  because of the inter-procedural context of this issue.  This example is quite simple, but where you can easily get lost is when that null value is coming from a long method call chain in multiple classes.

2.    Resource leaks

1  using System;
2  using System.IO;
3
4  namespace LeakExample
5  {
6      class Test
7      {
8          public String Run(String name)
9          {
10             StreamReader reader = new StreamReader(name);
11             String result = reader.ReadToEnd();
12             reader.Close();
13             return result;
14         }
15     }
16 }

I think for most of us this is pretty easy to see.   An object of ‘StreamReader’ type is allocated and its reference is assigned to the ‘reader’ member. If the call to ‘ReadToEnd’ throws an exception, control is transferred out of method ‘Run’, variable ‘reader’ goes out of scope, the object referenced by it becomes lost, but related resources are not disposed.  See my previous blog post on how some people miss this and what you need to do to fix this.

3.    Forward null pointer exception

1                  public class A {
2                      public void foo() {
3                          A a = new A();
4                          if (a == 0)
5                              if (flag)
6                                  a.foo();
7                      }
8                      private int flag;
9                  }

This is another example of a null pointer exception with a twist.  In this example the class data member ‘a’ is compared with 0 value at line 4, and therefore may still be expected to be null when it is dereferenced at line 6.

4.    Reverse null pointer exception

1                  public class A {
2                      public void foo() {
3                          A a = null;
4                          a.foo();
5                          if (a == null)
6                              a = new A();
7                      }
8                  }

Yes, another null pointer exception (getting a theme here?).  Same idea at #3 but in reverse order.  Here they dereference the data member ‘a’ at line 4 but later check for null at line 5.

5.    Empty catch clause

1  class FileHandler {
2      public void Open(String name) {
3          try {
4              // opening file ...
5          } catch (FileNotFoundException e) {   // defect - no statements in the 'catch' clause
6          }
7      }
8  }

Okay, so not exactly the big impact as the top 4, but I just had to mention it.  I’m always guaranteed to see some of these.  Granted they are maintainability issues more than anything else, but c’mon they are there for a purpose.
As promised, I did say that I would post the Java version soon.  It is next…