4 posts

Archive for March, 2009


JSR 305: a silver bullet or not a bullet at all?

Posted by Mikhail Ksenzov   March 30th, 2009

JSR-305 is a Java Specification Request intended to improve the effectiveness of static analysis tools operating in Java 5+ environments. The idea here is that one can use special purpose annotations in order to provide static analysis tools with hints regarding the behaviour and side effects of methods.

An example of such annotations can be found in the presentation ‘Annotations for Software Defect Detection’ by William Pugh, who is masterminding the whole spec. Here we go:

 1: void test() {
 2:    if (spec != null) fFragments.add(spec);
 3:    if (isComplete(spec)) fPreferences.add(spec);
 4: }
 6:
 5: boolean isComplete(AnnotationPreferences spec) {
 6:    return spec.getColorPreferenceKey() != null
 7:        && spec.getColorPreferenceValue() != null
 8:        && spec.getTextPreferenceKey() != null
 9:        && spec.getOverviewRulerPreferenceKey() != null;
10: }

What’s wrong with the snippet above? Well, the check for null on line 2 shows that the developer expects that the value of ‘spec’ can potentially be null, but it is still passed to method ‘isComplete’. Later, ‘isComplete’ attempts to dereference the value, which causes a NullPointerException.

According to Dr. Pugh, the best way to detect this issue statically is to force a developer to add the annotation @Nonnull to the method signature like this:

5: boolean isComplete(@Nonnull AnnotationPreferences spec) {

In this way, a basic static analysis tool that can minimally track ‘spec’ as a potential suspect for ‘null’ can issue a warning when the @Nonnull annotation is contradicted (that is, when ‘spec’ is passed to ‘isComplete’ as a parameter).

There are two problems with this approach:

  • it forces the developer to do work that rightly should be performed by a static analysis engine
  • it takes time to write the annotation for static analysis, but it takes even more effort to maintain the annotations and the actual code base in a consistent state.

In reality, the proposals behind JSR-305 exist to enable a tool intended for single function analysis (so-called intra-procedural analysis) to act as if it were performing whole program analysis by requiring the developer to state expected behaviour up front (whether or not that behaviour is actually expressed correctly in the developer’s code).

In contrast, this same scenario is supported by a whole program static analysis tool (so-called inter-procedural analysis) without developer intervention:

  1. First, a complete call-graph of the system is built, and then all the methods are ordered so that called methods are processed prior to callers — such an ordering allows the tool to generate all the necessary information about, in this instance, the method ‘isComplete’ by the time the analysis of the method ‘test’ begins.
  2. During the analysis of ‘isComplete’, the tool records the fact that the incoming argument ‘spec’ is dereferenced.
  3. Next, the method ‘test’ is analyzed. In this method, the variable ‘spec’ is checked for null, so it is tracked as a potential suspect for an exception. Using the information generated about ‘isComplete’ the tool can reliably issue a warning on line 3, since it already knows that ‘isComplete’ dereferences the incoming argument.

So that example applies to a simple unconditional dereference scenario. In more complicated cases, Dr. Pugh proposes to use the annotation parameter ‘when’, with one of the following values:

  • ALWAYS
  • NEVER
  • MAYBE
  • UNKNOWN

For example: ‘@Nonnull(when=When.NEVER)’ means that a value is always null in the given context.

This specification seems to be a compromise between the amount of information provided by a developer to a static analysis tool and the amount of effort a developer has to put into it, a compromise that does not seem to be a particularly good solution here. First of all, the amount of information provided in such a manner is insufficient to provide accurate analysis, and secondly this seems to be too much work for the developer, especially when this work can be avoided.

Let’s examine how conditional value dereferencing is supported by whole program static analysis tools:

 1: void test() {
 2:     entity.qualifiedName = null;
 3:     saveName(entity, false);
 4: }
 5:
 6: boolean    saveName(Entity entity, boolean qualified) {
 7:    String name;
 8:    if (qualified)
 9:        name = entity.qualifiedName.trim();
10:    else
11:        name = entity.name.trim()
12:
13:     save(name);
14: }

In this example, an inter-procedural static analysis tool would first analyze the method ‘saveName’. A good analysis engine should be able to record the fact that this method only dereferences ‘entity.qualifiedName’ if the second parameter, ‘qualified’, is set to ‘true’. This, it would appear, is a deal more detailed than one can practically achieve by adding @Nonnull(when=When.XXX) annotations, even with all the work the annotation implies for the developer.

Next, the method ‘test’ would be analyzed. A good static analysis tool will naturally keep track of ‘entity.qualifiedName’ because of the assignment to ‘null’ on line 2 and its therefore potential use in an exception causing context. However, given that the actual call to ‘saveName’ on line 3 uses ‘false’ as its second argument, such a tool will not issue a warning that would in reality be a false positive, since the knowledge gained from analyzing ‘saveName’ disqualifies any potential warning due to the conditional relationship between arguments.

In summary, JSR-305 proposes a whole roster of interesting ideas for using annotations to enhance static analysis of Java code, and NPE detection seems to be only one aspect of this specification request. In upcoming blog posts, we shall continue the discussion of proposed annotations as well as offering our own ideas about how and when annotations should be used in static analysis.


Now’s the time to invest in developer productivity.

Posted by Mike Laginski   March 24th, 2009

As software managers you’re undoubtedly being asked to do more with less in this economy. With companies continuously being forced to cut costs, the first shoe to drop is when you are told you need to cut headcount.

The second shoe drops the day after the painful deed is done and you look into the eyes of the team members that are left behind and try to put a positive spin on your world – their world. And that is when reality really hits home.  Less people, same number of problems.  No one “downsized” the backlog of customer requests, the bugs, the schedule expectations or the previous team’s workload.

At this point two groups form; the group of managers that simply puts their head down and grinds it out until things turn around (hoping things don’t get worse…which is really a do nothing strategy and those rarely work)…or the group that decides to be bold and innovative.  The natural inclination is to say the latter approach is too risky but in reality it is actually less risky, just more visible and more likely to be positively received by your team and your management.

The dev organizations best positioned to come out of this economic downturn stronger, are the ones with dev leaders that are focused on how to do things differently.  Agile development and further process automation with advanced tools become the mechanism to strongly position these dev teams for the better days ahead.  Why? Because just like every bubble, every downturn eventually ends.  As a dev manager, your real focus needs to be on what you want your team and your company to look like coming out of the downturn – heads down, battered and bruised but glad to be alive -  or lean and mean supported by a finely automated dev infrastructure and ready to capitalize on new opportunities.

By focusing on new approaches and automation, you  are helping your team feel they can get in front of the workload they have been presented with during these very challenging economic times. Automation is critical.  Tools such as continuous integration, refactoring, and code analysis all help eliminate wasteful, demoralizing “redo’s” of stupid mistakes they probably would not have made if they were not so maxed out, or if they were more familiar with the latest project you had no choice but to drop on their lap.  They see a way to spend more time on interesting, and challenging, innovation rather than just constant debugging.

“Hunker down” seems to be the mantra of our times, but “hunker down smart” and you and your team will be more readily positioned for the better days ahead.


Avionics Software Development and DO-178B

Posted by Brendan Harrison   March 18th, 2009

Today, I had a chance to connect with Connie Beane, the Director of Certification and Safety Critical Software for ENEA Embedded Technology, Inc. Connie has a deep background in safety-critical avionics systems development as a Federal Aviation Administration Designated Engineering Representative (DER) with authority for design assurance level A systems, software and complex electronic hardware. Her additional experience includes 12 years with the FAA in the Transport Airplane Directorate as a Project Officer, Federal representative and Secretary of the RTCA committee SC-180, which produced DO-254, as well as 8 years at Boeing as a Lead Engineer in Boeing Commercial and Boeing Aerospace Divisions.

I wanted to talk to her more about avionics software development, specifically give Kloctalk readers some additional background on the DO-178B guidance that’s used in avionics software development.

[Brendan]:  Can you describe the short version of the history behind this guidance, its purpose, and use in the industry?
[Connie]: In the early 1990’s Boeing was developing the 777 aircraft, which included very software intensive systems.  As a result, industry began using software development and verification tools as a means of developing software more efficiently. The FAA saw this trend as a concern since many of these tools were being used with very little human oversight. When DO-178B was published in 1992, a section on Tool Qualification was included.

[Brendan]:  Is DO-178B guidance only used by teams developing software for commercial aircraft or is it used elsewhere?
[Connie]: DO-178 was developed for commercial aircraft, however, today DO-178B is being used for much more.  It is being used for commercial systems which aid aircraft such as ground based navigation and communication systems. Interestingly, it’s also now being used in military applications both airborne and ground based.

[Brendan]: What’s the most common misunderstanding people have about DO-178B?
[Connie]: That it involves much more than good engineering practices.  DO-178B was developed by government and industry.  They incorporated good engineering practices and some additional activities to ensure safe, reliable software.

[Brendan]: What are a few of the core software development best practices that are required by DO-178B?
[Connie]: Quality Assurance audits, requirements traceability, robustness testing, standards for design, coding and verification.

[Brendan]:  Sounds like a pretty rigorous approach to software development, which of course is a good thing given its safety-critical nature. What kind of cost burden does DO-178B add to a typical project?
[Connie]: That depends on whether the software being developed is performing highly critical functions or not.  For highly critical software, the cost of developing to DO-178B could double the average development cost.

[Brendan]:  Switching gears a bit to your role at ENEA. Out of the full range of DO-178B service offerings you provide, what’s the most common problem you’re brought in to solve?
[Connie]:  Probably the most common issue is a project that is behind schedule and needs to be certified to meet a customer deadline.

[Brendan]:  What’s the biggest change you’ve seen in aircraft systems development during your career?
[Connie]:  Growth in use of software and complex electronic hardware to develop highly integrated avionics systems.

[Brendan]: What do you think the future holds for this guidance? Do you see it evolving in a particular direction that’s different from where it is now?
[Connie]: There is an international committee currently working on the next version of DO-178. This committee is addressing issues such a object-oriented design, model based development, and further defining qualification of software tools.  This committee is also developing a companion guideline to DO-178 which will be applied to ground based software.

[Brendan]: Any final thoughts?
[Connie]:  Companies resist embracing DO-178.  However, once a company has the processes and procedures in place to ensure compliance to DO-178, their software development process is easily understood, repeatable, and maintainable.  This translates to software which is reliable, understandable, traceable and maintainable.


Using Iteration Offsets in Agile Development

Posted by Todd Landry   March 9th, 2009

I thought for today’s topic I would delve into something that many organizations have to confront when moving to Agile that is, how to structure their iterations. Many organizations will find that iterations work better if development, testing, and documentation are all done within the same iteration period. Other organizations prefer to offset the testing and/or documentation ½ or a full iteration period after the development is complete. Having lived through both, I thought it might be useful to briefly list the good, and not so good of iteration offsetting.

First the good:

* Testing and documentation are not given features that have been completed at 5 pm on the last day of an iteration, and then expected to have their work done by the following Monday.
* Many dev features are totally complete prior to documentation or testing, so the doc and test teams can start their iteration running, rather than crawling.
* The documentation team is provided with some insight as to the “big picture” of a feature (and not just a standalone story in an iteration) and can plan their documentation for those features with that information in mind. This could help reduce rework on their part.

Now the not so good:

* Development can be drawn back into a previous iteration for bug fixes.
* If demo’ing iterations is a standard part of your process, it could contain a number of bugs since it hasn’t been through any formal testing yet.
* Communication is fragmented…some teams talking about Iteration x, some about Iteration x-1.
* Possible confusion, and more difficult to manage since there are multiple iterations on the go.

Is this list complete? Probably not…but hopefully it will give you something to think about if you’re either starting Agile development, or are struggling with your current implementation. For many projects, the negatives outweigh the positives on this list. As a Product Manager, I’ve seen that many situations where offsets as described above just don’t work. If that’s the case for you, then one possible approach that I’ve seen work is to implement a code freeze 3 days prior to the end of an iteration, and the developers then become part of the testing team.

My personal preference having worked in both scenarios, is to have dev, test, and documentation all occur within the same iteration. Now, my reason is somewhat selfish…when an iteration finishes, I want to show what we’ve done with some level of comfort of the quality and then be able to close the book on that iteration at that point. If that is the approach you prefer, here is a good blog about how to set up such an iteration. Of course, as with most things Agile, there’s never a “right” answer; it’s about finding the right iteration methodology that works for your team’s culture and project goals.