But I guess that all of these defects are quite rare, and one must take a start from a true-to-life defects, not just apply an approach to statically-typed languages here.

It just shows what is going to come down the line when it comes to analysis tools.

Personally, the next big thing is going to be design by contract. DBC opens up software for lots of things. Interestingly it reduces the need for unit tests. After all, if the code is being tested for all cases you come across, rather than the handfull that are in unit tests, you are going to get better code as a result.

Combine that with Pex where it uses the debugging library to trace tests, and then uses this analysis to find cases that will test other paths through your code, and you are running at a level of testing far above that which can be achieved by hand

I am not even too concerned about things like eval or calling functions in other clever ways that would not be caught by static analyzers. If a tool caught the normal cases that would still be a big win.

Doing a “proper” static analysis tool + type inference in Ruby isn’t *impossible* but it would require the user to make a lot of promises to the checker about what you do (or don’t do).

For example, in the presence of a single eval() that takes a user provided string in a Ruby program, you *can’t* statically determine what type an object has, or what method a class has, or how many arguments (and of what type) a method takes. Effectively the world becomes almost entirely unknown past that point. The way around that is for the developer to effectively say “yeah, but we only ever feed this eval() code that is restricted to these types of effects: A, B, C”

The same is the case for a lot of things, such as runtime “require” of files based on a string that may not be possible to statically determine.

Handling that kind of user annotation adds a *lot* of complexity. You can of course provide reasonable defaults. E.g. it’s reasonable 99.9% of the time to assume nobody will add instance variables to FixNum’s, or overload the “+” method of String to do something other than concatenation, even in the presence of eval().

But there’s a big grey area of things that may not be common, but that happens. Getting a Ruby-tool like this as comprehensive as one for C is a *huge* task.

It would be great if you could break into a running program and do some type analysis for dynamic languages.

- Paddy.

And, I’m certainly not trying to judge :) I’d like see if Ruby can break yet another stereotype and re-define program analysis (e.g. the way it redefined Web Frameworks). It is particularly interesting how Ruby error detection tools would approach the infamous “false positive” problem – an inherent weakness of all static analysis tools, when a real error/flaw could sometimes be buried under a pile of falsely reported issues.

BTW, have you tried Reek on some large Ruby codebases, like Rails? How many code smells were found? Got any interesting ideas on how to triage the detected smells?

But I think the main thing to note is that a number of such tools have sprung up for Ruby in the last year, possibly indicating a lot about the maturing of the Ruby community.