0 post

Posts Tagged ‘java’


Another resource leak

Posted by Alen Zukich   March 1st, 2011

It happened again.  For what seems like the 100th time, someone reports to me that they are seeing a number of false positive reports on the resource leak checker.  For those not familiar with a resource leak, take a look at a previous post.  Although resource leaks apply across most languages, the place where this question keeps coming  up seems to always be in Java or C# code.  My last query came from Java code, so we will use that as an example.  Here was a report where the FileInputSteam is not closed on exit.

FileChannel sch = new FileInputStream(...);
FileChannel dch = new FileOutputStream(...);
try{
   ...
} catch (...) {
   ...
} finally {
   try {
      sch.close();
   }
}
...

Do you see this issue?  Clearly, they close the FileInputStream in the finally block.  The problem is that you can still throw an exception when you try to create a FileOutputStream.  The simple fix is to encompass the FileInputStream and FileOutputStream in the try/catch blocks.

Static analysis tools are great for finding bugs like this resource leak.  In this case, the report that a FileInputStream is not closed on exit wasn’t enough information to debug this.  This brings us to a very important discussion with static analysis tools–trace.

Gone are the days where a static analysis tool reports a bug at line X with no context.  Today you get detailed traces.  The trace contains conditions and assignments to give you clues about why the tool is identifying a bug.  In this example below, identifying the “source” and “sink” of any defect will help you trace through the start/end of the control flow.


More importantly in the case of Java, here we have a clear understanding where exceptions are thrown.  This would have helped tremendously for the example above.



Detailed traces are very important and can do many things for individual checkers.  So whatever you do, don’t just look at the bug description, pay attention to the trace!


A new Android freshly baked!

Posted by Vahid Jozi   December 23rd, 2010

In my last post, I talked about some of the reasons why I started developing Android apps. The new Android 2.3 Gingerbread is even better than Froyo. I’m talking about a whole new Nerdy and the Droid factory.

With Gingerbread released and rolled out on the new flagship phone, the Google Nexus S, the developers’ wait is over. Some of the new features and functionalities baked within Gingerbread are:

1.    More power under the hood

My Nexus One lasts me about a day running Froyo. Android is now way smarter in managing its electrons with Gingerbread. With new interface design, smart process management and improved power management, it has increased the battery life significantly.

2.    Near Field Communication

NFC provides high-speed, short-range wireless communication capabilities and is responsible for most of the Gingerbread hype. NFC has introduced many new playgrounds for Android developers such as mobile tickets, mobile payment, identity documents, smart posters, electronic keys and more.

3.    Improved audio and graphics

Gingerbread now supports extra-large screen sizes and resolutions in addition to audio, graphic and input enhancements. This is a red carpet invitation for game developers to knock themselves out!

4.    Session Initiation Protocol VOIP telephony

SIP VOIP allows enhanced video and audio calling. It also supports video conferencing, streaming multimedia distribution, instant messaging, file transfer and online games.

5.    Native sensor support

Gingerbread now natively supports sensors such as the gyroscopes and barometers.

And let’s not forget new features like concurrent garbage collection, multi-touch software keyboard, enhanced support for native code, multi-camera support and new download management.

With all these new building blocks, I can’t wait to see what new apps are going to show up on my App Market!


Happy coding!


Why Android is such a developer magnet!

Posted by Vahid Jozi   December 21st, 2010


The open-source, Linux-based and hardware-independent Android mobile OS, with the new Android 2.3 Gingerbread release is giving mobile developers a whole new ball court to play in. Android is the fastest growing mobile OS among its competitors and with its share in the Smartphone user market growing, Android is attracting more and more enthusiastic developers.

Being a Java developer I jumped right into Android development about a year ago. There is a whole list of reasons why I chose to develop Android apps over other platforms and here are some of them:

1.    Low development costs and high returns

There is almost no cost to develop an Android app. There are no required licenses, specific IDEs or limited distribution channels. You may end up spending money on development and testing expertise, use of specific app stores and the purchase test devices, but the sum of all that is still a fraction of what you would pay to develop on other popular mobile platforms.

2.    Open and free

The two words every developer wants to hear are ‘Open’ and ‘Free’. Now you can have that on your mobile! This baby is license and royalty free because its underlying SDK architecture remains open source. The entire environment is available for customization, and developers and organizations can provide as detailed feedback as they want to the Android development team and watch as their issues get addressed in frequent new releases.

3.    Various distribution channels

Developers are not limited to the Android app market or any other channels to get their apps on consumers’ phones. Apps can be legally downloaded and installed from any unknown sources.

4.    The good old Java

Being Java based, Android uses a rich pool of libraries that provide extreme flexibility and room for uncapped creativity. Android’s topnotch documentation means virtually anyone with a working knowledge of Java can get Android applications off the ground.

5.    Flash Love!

Before Android gathered its well deserved power and popularity, Flash was thought to be a dying technology since it wasn’t supported by other mobile industry giants. There is obviously a ‘but’ here! The green robot embraced flash and took mobile web surfing to a whole new level.

6.    A little piece of the cloud

Android is heavily meshed with the cloud and it carries superior Google connectivity solutions. Google’s cloud tools are already soaring on the popularity list and they come right to the palms of the users’ hands with Android. One can also take advantage of browser connectivity solutions for Chrome and Firefox. In addition, Android provides open Bluetooth communication, something that is missing on some other popular platforms. Versions 2.2 Froyo and later allow the phone to become a portable hotspot. Now that is cool!


The awesome obviously doesn’t stop here. In my next post, I’ll talk about Android 2.3 Gingerbread features and what there is for us mobile app developers to play with!


Happy coding!


Android apps buggy?

Posted by Alen Zukich   December 22nd, 2009

We are starting to see a large amount of Android phones such as the Droid and Xperia X10 (see a review here) and the (soon-to-be-released) first Google phone, Nexus One. With this, expect the number of apps to increase significantly.

Droid vs. iPhone

Droid vs. iPhone

So with the increased number of apps, do these developers have the right tools to find and fix bugs? Take a look at the leader of phone applications-iPhone. There have been several posts (here and here) that recommend using the Clang static analyzer. Apple has taken it one step further, apparently rejecting iPhone apps that access private APIs. But Clang won’t help you with Java apps.

So what do the Android developers have? Android is just Java, so there are lots of tools, right? Certainly there are static analysis tools, profilers, unit testing tools and many more. But are these tools really taking into account the Android specifics?

Let’s take an example of a resource leak. Resources such as streams, connections and graphic objects must be explicitly closed; otherwise, you run the risk of throwing exceptions depending on the open resource.




For example:

1 static final String propertyFile = "my_config.ini";
2
3 static String getProperyFromConfigFile(String name)throws IOException {
4    Properties prop = new Properties();
5    FileInputStream st = new FileInputStream(propertyFile);
6    prop.load(st);
7    return prop.getProperty(name);
8 }

Here, a resource leak should be identified since line 5 opens up a FileInputStream, but is never closed before exiting the method. Now, this is all well and good and valuable to be found in any Android specific code, but what happens if I’m using built-in classes from the Android SDK?

For example:

1 public boolean onKeyDown(final int keyCode, final KeyEvent event) {
2    if (keyCode == KeyEvent.KEYCODE_DPAD_CENTER) {
3          final MediaPlayer player = MediaPlayer.create(this, ringtoneUri);
4          player.start();
5    }
6    return super.onKeyDown(keyCode, event);
7 }

Here, you have a situation where a MediaPlayer resource is created at line 3, but never closed on exit. Without the knowledge that MediaPlayer is a resource that should be closed, you will miss this type of issue. This extends to many resources and different issues. You can also have Android-specific null pointer exceptions and use of free issues.

Let me know if you’re doing Android development. I want to hear what you are doing to find these kinds of bugs.


Top 5 Java quality bugs

Posted by Alen Zukich   October 13th, 2009

In a previous posts I reviewed the Top 5 C/C++ and Top 5 C# quality bugs that I that I see time and time again looking at customer code. I wrote my Java Top 5 with an embedded programming focus and the folks at www.embedded.com decided to publish it on their site. Here’s a snippet below and the full Top 5 Java bugs article can be found here.

While C dominates as the programming language of choice for embedded development, the use of Java is definitely on the rise. In fact, according to a recent VDC survey, 12.3% of respondents currently use Java in the embedded space, and 17.9% expect to be using Java in the next two years.

For those transitioning from embedded development using C, you might find yourself falling into the hype that Java is a “safe” language. For example, Java developers face no requirement for managing memory associated with objects. However, this is where the trap may be laid. Even though there’s no need for memory management, developers may need to keep track of specific resources the object allocates. This is especially true in an embedded context where resources are often constrained. Even for experienced developers, these traps pop up time and again and can easily jeopardize your code quality and security.

Here’s a round-up of the top five programming issues developers should be aware of in embedded Java development [More...]


Findbugs not recognizing exceptions? Java static analysis

Posted by Alen Zukich   May 4th, 2009

We’ve posted previously on some of the differences between Findbugs’ open source Java analysis and commercial Java static analysis tools, specifically on the JSR-305 specification and source code versus byte code analysis topics. Due to these differences, many Java shops will use a commercial Java static analysis tool in conjunction with Findbugs to make sure they’re getting as complete issue detection as possible.

One area that’s been discussed previously is the ability to identify situations of possible null pointer dereference. This peaked my interest and led me to do some benchmarking against a few open source projects to assess the Findbugs analysis on intra- and inter-procedural possible null pointer dereference issues.

The normal assumption is that Findbugs is strong with intra-procedural analysis but unable to provide inter-procedural Java analysis. Here is an inter-procedural example from an open source project called hsqldb:

    protected void directRefreshTree() {

         int[]                  rowCounts;

        ...

            try {

                rowCounts = getRowCounts(tables, schemas);

            } catch (Exception e) {

                 //  Added: (weconsultants)@users

               CommonSwing.errorMessage(e);

            }

            ResultSet col;

            // For each table, build a tree node with interesting info

            for (int i = 0; i < tables.size(); i++) {

                col = null;

                String name;

                try {

                    name   = (String) tables.elementAt(i);

                    if (isOracle && name.startsWith("BIN$")) {

                        continue;

                        // Oracle Recyle Bin tables.

                        // Contains metacharacters which screw up metadata

                        // queries below.

                    }

                    schema = (String) schemas.elementAt(i);

                    String schemaname = "";

                    if (schema != null && showSchemas) {

                        schemaname = schema + '.';

                    }

                    String rowcount = displayRowCounts

                                      ? (" " + DECFMT.format(rowCounts[i]))

                                      : "";

    ...

    }

In the first try block getRowCounts() can have a null value.  How you ask?

    protected int[] getRowCounts(Vector inTable,

                                 Vector inSchema) throws Exception {

        if (!displayRowCounts) {

            return (null);

        }

      ...

    }

Okay, so automatically finding these issues is not going to happen with Findbugs.  I understand that, and that is why there are commercial tools to help with that.

But what really surprised me are the intra-procedural examples that are missed.  Again using examples from hsqldb and a specific class called TransferDb, Findbugs finds one issue. But there are clearly more intra-procedural issues that you would think Findbugs would have found.  Let’s take a look:

    TransferResultSet getData(String statement)

    throws DataAccessPointException {

        ResultSet rsData = null;

        try {

            if (srcStatement != null) {

                srcStatement.close();

            }

            srcStatement = conn.createStatement();

            rsData       = srcStatement.executeQuery(statement);

        } catch (SQLException e) {

            try {

                srcStatement.close();

            } catch (Exception e1) {}

            srcStatement = null;

            rsData       = null;

            throw new DataAccessPointException(e.getMessage());

        }

        return new TransferResultSet(rsData);

    }

This is a conditional situation where “srcStatement” could have a null value (the conditional is false), then you throw an exception with “createStatement()” and dereference later with “srcStatement.close()”.  Why does Findbugs miss this issue?  There are several issues of this type.

Or another situation where you throw an exception with “createStatement()” again.  This example spans multiple lines so you will find it in an attachment.  Here “select_rs” has a value of null and is clearly dereferenced much later on.

       ResultSet         col            = null;

        int               colnum         = 1;

  Statement         stmt           = null;

Source: null here ->  ResultSet         select_rs      = null;

        ResultSetMetaData select_rsmdata = null;

         try {

Exception throw here -> stmt           = conn.createStatement();

            select_rs      = stmt.executeQuery(TTable.Stmts.sSourceSelect);

            select_rsmdata = select_rs.getMetaData();

            col = meta.getColumns(TTable.Stmts.sDatabaseToConvert,

                                  TTable.Stmts.sSchema,

                                  TTable.Stmts.sSourceTable, null);

        } catch (SQLException eSchema) {

            // fredt - second try with null schema

            if (TTable.Stmts.sSchema.equals("")) {

                try {

                    col = meta.getColumns(TTable.Stmts.sDatabaseToConvert,

                                          null, TTable.Stmts.sSourceTable,

                                          null);

                } catch (SQLException eSchema1) {}

            }

        }

        try {

            while (col.next()) {

                String name = Dest.helper.formatIdentifier(col.getString(4));

                int    type        = col.getShort(5);

                String source      = col.getString(6);

                int    column_size = col.getInt(7);

                String DefaultVal  = col.getString(13);

                boolean rsmdata_NoNulls =

                    (select_rsmdata.isNullable(colnum)

                     == java.sql.DatabaseMetaData.columnNoNulls);

                boolean rsmdata_isAutoIncrement = false;

                try {

                    rsmdata_isAutoIncrement =

                        select_rsmdata.isAutoIncrement(colnum);

                } catch (SQLException e) {

                    rsmdata_isAutoIncrement = false;

                }

                int rsmdata_precision = select_rsmdata.getPrecision(colnum);

                int rsmdata_scale     = select_rsmdata.getScale(colnum);

                type = helper.convertFromType(type);

                type = Dest.helper.convertToType(type);

                Integer inttype  = new Integer(type);

                String  datatype = (String) TTable.hTypes.get(inttype);

                if (datatype == null) {

                    datatype = source;

                    tracer.trace("No mapping for type: " + name + " type: "

                                 + type + " source: " + source);

                }

                if (type == Types.NUMERIC) {

                    datatype += "(" + Integer.toString(rsmdata_precision);

                    if (rsmdata_scale > 0) {

                        datatype += "," + Integer.toString(rsmdata_scale);

                    }

                    datatype += ")";

                } else if (type == Types.CHAR) {

                    datatype += "(" + Integer.toString(column_size) + ")";

                } else if (rsmdata_isAutoIncrement) {

                    datatype = "SERIAL";

                }

                if (DefaultVal != null) {

                    if (type == Types.CHAR || type == Types.VARCHAR

                            || type == Types.LONGVARCHAR

                            || type == Types.BINARY || type == Types.DATE

                            || type == Types.TIME

                            || type == Types.TIMESTAMP) {

                        DefaultVal = "\'" + DefaultVal + "\'";

                    }

                    datatype += " DEFAULT " + DefaultVal;

                }

                if (rsmdata_NoNulls) {

                    datatype += " NOT NULL ";

                }

                v.addElement(inttype);

                datatype = helper.fixupColumnDefRead(TTable, select_rsmdata,

                                                     datatype, col, colnum);

                datatype = Dest.helper.fixupColumnDefWrite(TTable,

                        select_rsmdata, datatype, col, colnum);

                create += name + " " + datatype + ",";

                insert += "?,";

                colnum++;

            }

Sink: dereference here-> select_rs.close();

            stmt.close();

            col.close();

        } catch (SQLException e) {

            throw new DataAccessPointException(e.getMessage());

        }

So I’m a little confused as to why these issues are not highlighted.  It seems that this is the exact type of analysis Findbugs does.  Maybe it has to do with not recognizing the possible exceptions?  Does anyone have any ideas?


JSR 305: a silver bullet or not a bullet at all?

Posted by Mikhail Ksenzov   March 30th, 2009

JSR-305 is a Java Specification Request intended to improve the effectiveness of static analysis tools operating in Java 5+ environments. The idea here is that one can use special purpose annotations in order to provide static analysis tools with hints regarding the behaviour and side effects of methods.

An example of such annotations can be found in the presentation ‘Annotations for Software Defect Detection’ by William Pugh, who is masterminding the whole spec. Here we go:

 1: void test() {
 2:    if (spec != null) fFragments.add(spec);
 3:    if (isComplete(spec)) fPreferences.add(spec);
 4: }
 6:
 5: boolean isComplete(AnnotationPreferences spec) {
 6:    return spec.getColorPreferenceKey() != null
 7:        && spec.getColorPreferenceValue() != null
 8:        && spec.getTextPreferenceKey() != null
 9:        && spec.getOverviewRulerPreferenceKey() != null;
10: }

What’s wrong with the snippet above? Well, the check for null on line 2 shows that the developer expects that the value of ‘spec’ can potentially be null, but it is still passed to method ‘isComplete’. Later, ‘isComplete’ attempts to dereference the value, which causes a NullPointerException.

According to Dr. Pugh, the best way to detect this issue statically is to force a developer to add the annotation @Nonnull to the method signature like this:

5: boolean isComplete(@Nonnull AnnotationPreferences spec) {

In this way, a basic static analysis tool that can minimally track ‘spec’ as a potential suspect for ‘null’ can issue a warning when the @Nonnull annotation is contradicted (that is, when ‘spec’ is passed to ‘isComplete’ as a parameter).

There are two problems with this approach:

  • it forces the developer to do work that rightly should be performed by a static analysis engine
  • it takes time to write the annotation for static analysis, but it takes even more effort to maintain the annotations and the actual code base in a consistent state.

In reality, the proposals behind JSR-305 exist to enable a tool intended for single function analysis (so-called intra-procedural analysis) to act as if it were performing whole program analysis by requiring the developer to state expected behaviour up front (whether or not that behaviour is actually expressed correctly in the developer’s code).

In contrast, this same scenario is supported by a whole program static analysis tool (so-called inter-procedural analysis) without developer intervention:

  1. First, a complete call-graph of the system is built, and then all the methods are ordered so that called methods are processed prior to callers — such an ordering allows the tool to generate all the necessary information about, in this instance, the method ‘isComplete’ by the time the analysis of the method ‘test’ begins.
  2. During the analysis of ‘isComplete’, the tool records the fact that the incoming argument ‘spec’ is dereferenced.
  3. Next, the method ‘test’ is analyzed. In this method, the variable ‘spec’ is checked for null, so it is tracked as a potential suspect for an exception. Using the information generated about ‘isComplete’ the tool can reliably issue a warning on line 3, since it already knows that ‘isComplete’ dereferences the incoming argument.

So that example applies to a simple unconditional dereference scenario. In more complicated cases, Dr. Pugh proposes to use the annotation parameter ‘when’, with one of the following values:

  • ALWAYS
  • NEVER
  • MAYBE
  • UNKNOWN

For example: ‘@Nonnull(when=When.NEVER)’ means that a value is always null in the given context.

This specification seems to be a compromise between the amount of information provided by a developer to a static analysis tool and the amount of effort a developer has to put into it, a compromise that does not seem to be a particularly good solution here. First of all, the amount of information provided in such a manner is insufficient to provide accurate analysis, and secondly this seems to be too much work for the developer, especially when this work can be avoided.

Let’s examine how conditional value dereferencing is supported by whole program static analysis tools:

 1: void test() {
 2:     entity.qualifiedName = null;
 3:     saveName(entity, false);
 4: }
 5:
 6: boolean    saveName(Entity entity, boolean qualified) {
 7:    String name;
 8:    if (qualified)
 9:        name = entity.qualifiedName.trim();
10:    else
11:        name = entity.name.trim()
12:
13:     save(name);
14: }

In this example, an inter-procedural static analysis tool would first analyze the method ‘saveName’. A good analysis engine should be able to record the fact that this method only dereferences ‘entity.qualifiedName’ if the second parameter, ‘qualified’, is set to ‘true’. This, it would appear, is a deal more detailed than one can practically achieve by adding @Nonnull(when=When.XXX) annotations, even with all the work the annotation implies for the developer.

Next, the method ‘test’ would be analyzed. A good static analysis tool will naturally keep track of ‘entity.qualifiedName’ because of the assignment to ‘null’ on line 2 and its therefore potential use in an exception causing context. However, given that the actual call to ‘saveName’ on line 3 uses ‘false’ as its second argument, such a tool will not issue a warning that would in reality be a false positive, since the knowledge gained from analyzing ‘saveName’ disqualifies any potential warning due to the conditional relationship between arguments.

In summary, JSR-305 proposes a whole roster of interesting ideas for using annotations to enhance static analysis of Java code, and NPE detection seems to be only one aspect of this specification request. In upcoming blog posts, we shall continue the discussion of proposed annotations as well as offering our own ideas about how and when annotations should be used in static analysis.


Java source code vs bytecode analysis

Posted by Alen Zukich   January 6th, 2009

David posted an interesting discussion on the usage of static analysis tools by developers to find security vulnerabilities.  As always the discussion with static analysis tools lean towards the false positive and false negative discussion.  But also David mentions their results are sometimes difficult to understand.  

This is one of the reasons Klocwork switched from a bytecode analysis tool for Java to a source code analysis tool.  As both have their advantages and disadvantages (and I admit I’m very biased here) we have certainly found that we have been able to reduce our false positive rates, find more issues (not to mention add new issues quickly) and provide more details on the results.

The reason for this is that the bytecode started to get noisy in Java 5 and even more so in Java 6.  Specifically one of the great advantages that static analysis tools have today is being able to show you the details of any issue it finds.  For example

public class Source {
     public static Source getInstance() { return null; }
     public int getValue() { return 32; }
     public Source() { getInstance().getValue(); }
}

This is a Null Pointer Exception issue. Although this is a very small example it is an inter-procedural issue and something that is missed by other defect detection tools. The important thing here is to help the developer understand this issue by tracing through the paths that make this happen.  In other words you need to know that getInstance() has the potential to return null and what paths bring us to the actual error.  This can get quite complex where it starts to not only span the same class but different methods in different classes.

Because of this extra noise this could be part of the reason your seeing new Java annotations being introduced.   Many of you may be familiar with the Java annotation that were introduced in Java 5.  This was part of a specification called JSR-250.  There is a new specification called JSR 305 for the purposes of allowing annotations to help assist tools that detect software defects.  Interestingly enough it is Findbugs leading the charge to add this in for Java 7.  Interesting to see if this specification happens.


ISV software quality; tortology or oxymoron…

Posted by Gwyn Fisher   December 10th, 2008

It’s kind of bizarre, but in my pre-Klocwork experience of running ISV development groups, from small teams to global enterprises, it never struck me as wrong that we would routinely ship software containing critical bugs. We knew we were doing it. We knew, on some abstract underground never-to-be-admitted layer of our deepest darkest souls, that this was a “bad thing.”

But mostly, we knew that when somebody found a bug we could just send them a patch.

And what’s more, we knew that customers expected this behavior. We got requests to “send us a patch quickly, this is causing me {insert unpleasant business scenario here}.” We had customers telling us that we were a great company to deal with, because we responded to patch requests quickly.

We weren’t the bad kids on the block, we were right up there with the upstanding corporate citizens of the software world!

Why are software companies allowed, perhaps even expected, to mess up on this scale? If we were engineers, we’d be sued. If we were doctors, we’d be in court before you could say something snappy and relevant like “most software sucks.” But for ISV developers, hey, it’s just business.

Talk to developers in the embedded space, or mil/aero, or telecoms, or a bunch of other places where this approach to business isn’t acceptable and it’s like talking to engineers. They understand what their tolerances are, they understand what it takes to make quality software, and they fundamentally understand on a basic level what happens to them and their company if they get it wrong. Inventory turnover is one thing, killing people is something else.

So is it just that ISV software is unimportant? That ISV software won’t ever be put in a situation where it could be life-threatening (or life-enhancing), so it doesn’t matter if it crashes all the time?

Obviously that’s not the case. Software created by ISVs is used all over the world, in all manner of situations, some dire, some not, but at the end of the day, in every situation, there’s a name on that software.

Your name.