As new words and concepts diffuse in to wider use, their definitions become simpler or broaden to cover more scope.  Like the kid’s telephone game, each time the concept is passed to another developer, the information gets a little more muddled. In software development, declaration, macros, syntax and other programming constructs have to be exact or the compilers will fail.  Yet, when developers discuss concepts about programming, most of the time, that precision of language is lost.

The telephone game  seems to have happened to refactoring.  I subscribe to what would be consider the  ”classic” definition of  refactoring: the process of optimizing or extending a class  but leaving the existing exposed interface alone.  It seems that refactoring has been generalized from that definition into covering all activities related to touching old code.  Even worse, it has become an excuse to rework someone else’s code and then bitch about how bad it was.  Just look at the twitter stream for refactoring for a never-ending torrent of  abuse.

There is one simple way to tell if your efforts are truly a refactoring or not.  Did you break any of the unit tests?  If you did, then you are rewriting code instead of refactoring it and you had better update all the test cases while you’re at it.  Don’t get me wrong, there are times to rewrite software, but they are few and far between and, in my experience, it almost never pays off.

Refactoring isn’t just a nice philosophical idea; it supports one the most basic concepts in software development–backwards compatibility.  If you want to keep your customers and enjoy that paycheck, don’t break features or APIs in products without good reason or  get buy-in before release day.

To refactor the medical phrase “First, do no harm“, for developers it should be “First, break no test”.

Most code reviews are done in-person, 60%  according to data  from a Forrester Consulting study commissioned by Klocwork.  So how do you accommodate remote sites, out-of-office employees  or off-shore development shops?

Most software developer teams will face some form of remote development challenge during their careers or product cycles.  As demonstrated from the data above, the breakdown of remote need is as follows:

• 76% use some form of outsourcing,
• 64% have some developers  located outside of the main campus,
• 40% of reviews are conducted with remote participants.

You can’t let development come to a grinding halt simply because a critical team member is not physically available at the scheduled time or location.  For most organizations, code reviews need to be performed and employee travel is not the solution for cost and timing reasons.  This has driven the adoption of lightweight review processes and new tools that support it.

Klocwork built a code review tool for this express purpose.  Other ones exist like Code Collaborator and the open source Review Board .  How do you support your remote code reviews?  Email?  Wiki? Or a purpose-built tool like one of the ones mentioned?

I love this quote by Carl Ek from  Code Integrity solutions:

There are 0010 0000 kinds of people in the world: Those that understand the difference between Big Endian and Little Endian, and those that do not.

Issues with Endianism and processor architecture ports are becoming more and more common these days as more desktop source code moves into different arenas.  Gone are the days when the 32-bit memory model or little-endian format dominate. Software changes are required to support the growth occurring not at the desktop, but in the server  and mobile platforms.

Mobile devices especially have opened a Pandora’s box of Endian and memory problems, with variety of processor architectures with ARM^[1] leading the way.  Add to this mix,  end-consumers are demanding desktop features like Adobe Flash or Office apps on mobile devices, many a stable codebase will fall apart when ported to either mobile or server.

For developers porting to different platforms, there are some significant challenges.  Just to list a few:

• CPU optimizations need to be reviewed
• inline assembly calls require rewriting or removal
• machine word (WORD) allocations may require refactoring
• any binary data exchanged over the network stacks require verification

None of these are  new, they’re just not a common skillset for most developers.

Source code analysis can be a boon in two ways. Firstly, in the planning phase by helping you determine the breadth of the effort,  and secondly by identifying any existing  issues, particularly of the memory allocation and Endian varieties.

For more in depth information, there are two recent articles available from Dr. Dobbs:

• Porting to 64-bit Platforms by Irving Rabin
• Detecting Endian Issues with Static Analysis Tools by Carl Ek

^[1] Note: Some ARM processors support both big and little Endian formats.

So a while back, I explored where developers get their information.  Surprisingly, it is hard to find hard data on the subject.  As a bonus from a Forrester study commissioned by Klocwork into the habits of code review, part of  the data revealed developers’ use of social media tools.  When asked directly about their use of these tools to communicate with other developers, the majority polled would not choose a social media channel.

It just goes to show that yet again, software developers are a breed apart.  As an aside, as I was researching this topic, I found an interesting post on why Social Media Experts are poets, Software developers are novelist that delves into ideas on barrier-of-entry as related to quality-perception of creative tasks.

What is the ROI to the company for developer tools?  This has always been and continues to be a struggle in any development organization. Developer productivity to date has been very poorly measured and studied. Programming has always been a creative task bound within the constraints of  a framework.  Many people have tried to measure direct individual developer productivity with less than convincing results:

Bugs per dev, bugs per team, bug regression rates, bug trend lines, comparative .NET Framework book sales rates,  # of [static analysis] violations per kloc, etc, etc.. the list is really endless…

So when it comes to assessing the value of development tools, what are the motivational and productivity drivers?  During my time as a development manager, my focus was always on reducing roadblocks and mindless repetitive tasks.  When you are well compensating developers to use their grey matter, you don’t want their efforts to be wasted on the simple stuff.

Focusing on a bottom-up approach, I base the tool choices on the marginal cost for adding a developer.  After getting the basics out of the way, including corporate overhead, dev box, standard  software load and a complier, what’s next?

Well, there are a plethora of vendors all screaming at you to buy their product, but let’s take a step back and look at what makes a developer produce better code.

Here is my list in order and why.  It starts at the developer desktop and depends on frequency of use and then spreads out to the rest of the development team:

• Source control (with nightly backups) – Even for a development team of one, don’t leave home without it.  The benefits of source control really can’t be overstated: revision control, easy diffs, build integration, product branching  and maintenance, etc.

• static analysis – like grammar checking for Office but so much deeper. It has been proven time and time again that getting rid of the minor flaws and style inconsistencies right when the developer is working and the code is fresh in their brain is the most productive.  Waiting 2 hours, a  day and for some even a week costs a context switch.  When you add in the regression analysis for the maintainability, this tool just shines and pays for itself quickly

• Issue tracking – some might debate that this tool’s placement should be above static analysis, but as I said,  my focus here is on the developer’s desktop and not team or management needs.

• refactoring – only sightly better than find/search/replace - not essential but when used frequently and consistently can be used for good productivity benefits .  It’s one of the small tools that get you through the day.

• code review – similar to static analysis but now we have widened our view off of the desktop and include the team to build better code. No one person can know everything and extra pairs of eyes on the code is only going to make it better.

• unit testing – anything to take away the drudgery of building simple test cases that completely cover the class/api interfaces and maintaining with the refactoring that will happen over time.  You might ask why unit testing is lower on the list. Simple. You can’t test code that has not been written yet.

• dynamic/performance profiling and input injection – your customer will like you for this one.  Over time, most applications grow, add new features and become sluggish pigs.  Version performance testing and making critical judgments in trading off performance for value is one of the  keys to repeat customers.

There you have it.  A list of tools for developers (note I said developers not PMs or teams) that I consider essential to have on each desktop to get them through their day.

I just got back from a visit to the Valley and had an awesome week in San Jose/San Fran.  I even had time to play a bit of the tourist this time (I ran the Golden Gate bridge/Presidio).  All that was fun, but what I always enjoy is the conversations we had with customers and prospects at this year’s ESC SJ 2010 conference.

It is always interesting listening to their successes and teasing out the trending topics and new issues that matter to development teams.  Here are the top three themes that caught my ear this year:

1. The economic rebound is well underway, with growth and optimism from every quarter.  It may be too early to see results on the balance sheets, but the positive attitude is back.
2. Embedded developers are searching for enterprise-class developer productivity tools, like static analysis, for even tiny codebases (less than 40 kLOC).
3. By far, the most-often raised topic in one-on-one conversations was coding standards, with MISRA C and C++ as the favorite.  MISRA’s time has definitely arrived for the embedded community.

So all in all, a great time, and looking forward to next year.

There has been a start to bring the concepts of  lean manufacturing  into agile development. Recently, Mike Cottmeyer in How to Build a Large Agile Organization proposes that Agile on its own is not enough for a large organization.  In his view, Agile falls short and needs to be supplemented by additional methodologies like Lean or Kanban when coordinating outside the development team.

If adoption of Agile is impeded by its very nature in large organizations and Kanban is the proposed answer, then the Agile solution is insufficient. Agile needs to expand its scope to be relevant and useful for non-developers as well as across development teams.

To understand how Lean applies to Agile development, I’m going to take a short detour though history.

Mapping manufacturing principles to software development is an interesting cross-pollination of ideas. Discrete manufacturing is quite different from application development, but that doesn’t mean the software industry can’t learn a thing or two from a different sector.

Lean was born out of a need to re-invent the manufacturing industry, which had not really evolved since the inventions of Henry Ford and the production line. From Ford’s time to the post second world war period, most manufacturing was very good at making enormous quantities of the same product, regardless of the demand. Ford’s famous quote about color clearly exemplified the thinking of the day: “Any customer can have a car painted any colour that he wants so long as it is black”. In other words, Ford’s production line was optimized for manufacturing, not profit, and turned out to be quite inflexible when market conditions changed.

In the 1950s, Sakichi Toyoda made a revolutionary leap forward with two principles:

1. Pull vs. push – at any point in the production process, the trigger to start work on a production unit is governed by its upstream neighbor.  As an example, I do not start my work on a product unit until the guy following me says he will be able to receive it.
2. Efficient manufacturing depends on the management of three key inefficiencies: overburden (muri), inconsistency (mura), and eliminating waste (muda).

Together, these elements formed the underlying principles that Sakichi spearheaded into what is now known as The Toyota Production System (TPS). The TPS has subsequently been used as the the basis for Western derivatives such as Just-In-Time, value-stream mapping, Six Sigma and Lean, to name a few.

So what does this have to do with Agile and large organizations?

There are well-documented cases where agile alone was not enough, and that’s where Lean/TPS can add value.  For the most part though, the application of Lean principles has been limited to just one part: Kanban.

The TPS Kanban methodology has two aspects. First,  a Kanban card is attached to every unit under production and carries contextual information (metadata) about the tasks that need to be performed on that unit  and second, task readiness and data are used to trigger an specific action (work).

Over the past decade, the Agile methodology has been used successfully within  development teams, usually sized between  8 and 15 people. Agile’s benefits and values for this type of environment have been well articulated by many others (including on this blog), but most Agile adopters may not have realized the close mapping to Lean/TPS.

• Muri (overburden) – overproduction – in an Agile context, this is usually expressed as over-planning
• Mura (inconsistency) - elimination of bugs at the earliest stages, resulting in more  stable and reliable iterations
• Muda (waste) – close interaction with the customer to absorb change and prevent wasted iterations
• Kaizen (continuous improvement) – refactoring, unit testing, system integration

Secondly and more importantly for large teams, the TPS/Lean idea of pull vs. push is key. But there are other aspects of Lean/TPS that would benefit software development, Kanban being an important one but not the only one.

In an Agile context, Kanban is usually expressed as a board or wall with movable index cards to visualize units of customer value and work flow. This is where I think the rails have come off Agile/Kanban compared to the original TPS philosophy.  Kanban is just one gear in the whole TPS methodology.  Its an integral part but no more important than the other parts.  To function optimally, the TPS/Lean requires all the piece to be implemented not just one.

The other aspects of TPS/Lean are:

• Andon (signage, early warning)  - literally means paper lantern and is used to call attention to a problem in the process.  For Agile, it should be express as how do you measure your team’s progress and convey that information to the whole organization.
• Jidoka (autonomation) – automation with human intelligence.  The efficient use of tools like static analysis and continuous build to aid in development.
• Poka-yoke (fail-safing) – not just exception handling, but actual prevention of faults and counter-measure strategies to prevent the fault from reoccurring.

These other parts of the TPS were not born because people like more processes and rules; they came out of need, something the agile methodology has yet to realize it requires.

As Google Chrome climbs out of obscurity in the browser market and expands into a light-weight but fully functional OS, security seems to have become a top of mind issue over at chromium headquarters.

In the Chromium Blog, Chris Evans of Chrome Security announced a cash for bugs initiative, paying between 500 and 1337 USD depending on the severity for any previously undiscovered flaw.  I am glad to see Google encouraging the community at large to participate in hardening my current browser of choice.  As Chris points out, Mozilla was one of the first to embark on this type of program, but I am happy to see Chrome following suit.  Me and my online transactions appreciate it.

Hmm.  Maybe I should roll-up the sleeves and  invoke the  ”I’m gonna write me a minivan” approach and get the driveway cleared for the armored cash trucks.

But seriously, if you’re interested in helping out and getting a small reward for your efforts, visit the Chromium Security project.

I have always been fascinated by the whole area of code vulnerabilities and security exploits and how hackers turn those issues into real-world problems for the rest of us.

Jeremy Brown posted an interesting article on Jeremy’s Computer Security blog where he uses his security know-how to draw a straight line between a software vulnerability found with static analysis and a real 0day exploit on an open source project called gAlan.

Jeremy takes us on a short journey where he finds an unprotected buffer with static analysis, creates an exploit payload to cause a buffer overrun, rewrites the instruction pointer and executes a telnet session, demonstrating how easy it is to turn a run of the mill application into a tunnel into the OS.

One of my colleagues did a similar presentation like this a few years back with a Firefox vulnerability but this is a much better example! Very cool work Jeremy.

Enjoy…

I have always loved “I’m gonna write me a new minivan”  from Scott Adams.  To me, it never gets old.  Originally published in 1998, the theme that applied then still does today: driving 100% of defects or bugs out of the code-base is a laudable goal, but is it really the right one?   I would have to argue no.  There’s no silver bullet out there that will find all software defects and solve issues automagically, and until there is, software development will continue to struggle with prioritization.  Unfortunately, we live in a world of finite resources and constantly evolving demands, but we can always dream about being Wally for a little while.