The Industry

Rails Testing, TDD, BDD, and Competitive Advantage [Not]

Finally made it up to the two chapters on testing in the Obie Fernandez book ‘The Rails Way’. This is one of the best programming framework books I have read. Concise, clear, balanced. Up to now it was only the database chapters that were a wee bit presumptive about the depth of knowledge on the part of the reader - but still, quite good. Very, very few books do I read from start to, well, page 640 so far.

With that said, the testing chapters… are either uncharacteristically shallow, or… the testing in Rails sucks as much as JUnit and all the other toys people strut around giving each other group hugs about. Maybe there is a wonderful plugin out there that has so far slipped through the blogaratsi’s fingers and raises TestUnit and RSpec above what I have seen so far.

All these things are just test harnesses. Scaffolding and files in which to put and run your tests. Whoo Hoo! Tests which are usually so inane and simple that people do not have to think about what really needs testing.

Which is the point of course. All these methodologies are just dreamed up by people to make managers who do not know how to manage tech projects feel like they are doing something. I.E. they convince coders that they do not have to think, and managers that they do not have to think, and that they do not therefore need people who think on their team - and so can treat them and pay them like interchangeable ‘coder bots’ and ‘manager bots’ .

Why is not having to think so bad? Because this approach gets us track housing and ‘no child left behind’. Where the focus is all about passing tests and inspections, and not about the real reason why we are doing this, what the purpose of it all is. And it is the challenge of thinking about these real purposes and how to do them better that makes life worth living and the meeting and overcoming of these challenges that makes products worth using.

So now you got these test harnesses, that double the size of the code, are laborious to write, and are intricitly dependent on the code signature.

It is so hilarious… ever since UML [3 top-notch modelers who instead of making a wonderful uber modeling language, since, you know, they are experts at it, decide to make a visual language instead - and as far as I can see ignored all the previous work on that topic], all these methodologies that are accepted with such religious fervor reduce productivity by as much as an order of magnitude. But.. it makes managers and consultants happy, and it is these people who drive our industry… so I am sure there is no end to newer, bigger, better[better names that is. We got some good names already… EXTREME, AGILE … even RATIONAL, much better than ‘top-down’ or ‘bottom-up’ or ‘functional decomposition’ or ‘piecewise refinement’ huh? :-)] methodologies in sight.

What a testing framework should do, IMHO, [for DB-driven apps, which helps makes things easier. The view, the view is a little more work], is to:

0. Construct the DB from scratch with several / many DB building operations [controller methods].

1. Examine the DB and application operation. If things are good, save the state of the DB and use it to compare against when test 0 is run again.

2. Take a DB in several initial conditions [empty was tested in step 0], with several objects/rows/tables in boundary conditions.

3. Permute the DB using a sequence of 1..n standard operations [controller methods]

4. Checks the state of the DB [this still may require tight coupling, depends…]

5. Permute the DB with the intent to break it using a sequence of 1..n standard operations with invalid parameters

6. Check the results [theoretically the DB should not have changed]

In detail:

This might be actually easier in Java, with reflection and static typing IF THEY WOULD ADD TYPEDEFS [I have finally given up], but the idea is that each method and the types and ranges of its arguments are declaratively described:

method_declaration :add_user([{:name => ‘name’, :type => String, :min-length: 6, etc…. :good => [’sam’, ‘joe’], :bad => [’%$%’], :boundary => [’123456′], :db_attack => [’select * from user’], view_attack => [’‘… }])

OK. I am not very good with the attack code thing, not spending much time on that side of the hobby.

And the test driver takes these declarations and calls the method with good, bad, attack, boundary condition and insufficient parameters and evaluates the results [some of which must be eval’d by hand coded methods].

Being lazy, test drivers can often use the type of the parameter [email address, uri, image, password, phonenumber, enum, etc] and figures out for itself the good, bad, etc values for the method parameters. Sometimes this can be more difficult for some types, and things like countrynames, email addresses, image uris, enums, etc. the list of ‘good’ values must be supplied by a table.

The driver can also ‘SANITY CHECK’ the database using tables described by these highly described parameters, along with special hand-coded sanity checking procedures. And runtime sanity checks and corresponding SANITY REPAIR can deal with those ‘acts of god and users’ that might otherwise require, say, a slow transaction-based DB.

Speshy [the slow SOB] uses this technique, and in the Obsidian Framework every method is a plug in, and each parameter and return value to each method is a fully described parameter, and the test framework just kind of writes itself. The older Cadabra has a class-based [metaclass actually, being a metaclass-driven prototype-instance-based framework in Java] approach and we’ll list it here [relations in Cadabra, like delegate, inherit, etc., are first class objects]:

//——————————–
// Validate methods
//——————————–
public String validate();
public String validateValue(String attributeName, String attributeValue);
public String validateRelation(String relationType, CaiObject src, CaiObject dest);

//——————————–
// Test methods
//——————————–
// Test internal behavior of this object
public CaTestResults selfTest(CaTestConfiguration config);

// Check current internal values/setup
public CaTestResults selfCheck(CaTestConfiguration config);

// Test external behavior of this object
// i.e. send itself msgs and att values
public CaTestResults unitTest(CaTestConfiguration config);

// Check current relationships of this
// object to the rest of the system
public CaTestResults unitCheck(CaTestConfiguration config);

There seems to be an obsession, for some reason, in the Rails community with mocking the database - one method at a time. The only reason I can see to mock the DB, with my extremely cloudy and impatient glasses, is when the schema is generated by an organization separate from the one implementing the application, and that organization is a little sloooow [I am being so kind in this post today]. The happened on a Raynet / Ericcson project in 93-94 (95?) and, using a rigid layering architecture, we mocked the entire DB, generating [pseudo - it needs to be repeatable for bug fixing] random but semantically valid data to test the app with. A Rails tool could easily do this by scanning the DB and generating mock tables and rows on startup, in memory, and then overriding the DB find() methods to return this mock data. No more hand writing each and every mock. Ugh. [Ugh, not because it is hard, but because it is so BORING!]

Well, as you may be able to tell, I am kind of disappointed. This is the first large hole in Rails [as a web dev platform] that I have found so far. Kind of surprised that it is here, in TEST of all places, with all the TDD this and BDD that.