So, a couple of weeks ago Obie Fernandez proposed some kind of Rails Maturity Model (RMM) by analogy to the more generic CMM model for generic software development.
For his troubles, Obie got a lot of guff from the peanut gallery, much of it from developers who have worked at companies where CMM is in place and know what a nightmare that can be.
I suspect this isn’t the last time this will come up in this post, but I did work at a major telecommunications company that pushed many of it’s software divisions to CMM Level 5. We really did have a process in place for evaluating proposes to improve processes. I can’t remember whether we actually had a meeting about reducing the number of meetings, but it was definitely proposed…
My general response to any kind of measurement proposal about programmer or team quality has two points:
- I absolutely understand the problem, and the desire on the part of those of us who think we do good work to be able to point to a metric that validates us. Or, best case, shows us how do even better work.
- I’m very skeptical of any specific attempt to measure individual productivity or team maturity.
The Problem
All of us, as consultants, spend a great deal of time explaining to our clients our best practices for software development and how we are going to get them the maximum amount of features as efficiently as possible. We throw around terms like “Agile”, “Scope Creep”, “Test-Driven Development”, and the like, and hope we explain them convincingly.
In the end, though, the client takes a leap of faith in evaluating our credentials and professionalism.
It’d be great to be able to point to something, somewhere, that independently and incontrovertibly says that we are awesome.
This problem is not limited to us, the same issue applies more-or-less acutely to any professional. How can I, as a layman, really evaluate whether a doctor is skilled, or a lawyer, or an accountant? Some of these professions have organizations that purport to evaluate members in one way or another, but I don’t know of one of them that’s considered really useful.
But I Don’t Have A Solution
That’s the problem. This would be the point where I propose my brilliant solution. As I said, though, I’m skeptical of any specific solution having beneficial effects on the Rails community as a whole.
The only test of team quality that I’ve ever been able to even slightly get behind is Joel Spolsky’s Joel Test, which is just twelve yes-or-no questions. It’s not perfectly applicable to all situations, but it is really quick and easy, and sort of points to areas of improvement. But, it doesn’t really cover Agile processes, and it’s not like we can tell a client that we’re going to do good work for them because we score a 10 on the Joel Test.
There’s a tension here between bottom-up and top-down. Most of the tweaks and changes to our process at Pathfinder come internally to continuously improve our own work based on our own experiences. To the extent those processes are based on local issues, they may not be applicable to other teams. At the same time, there’s a top-down desire to add practices from the Joel Test, or an RMM model, or some other process guide, because those process have worked in other places, even though they may or may not be applicable here.
Parenthetically, as far as I could tell from my little corner of the major telecommunications company, the CMM-imposed processes had no affect at all on development. The processes that were put in place that weren’t solving problems were generally ignored, with some leaf covering to satisfy paperwork requirements (gaming metrics was a very popular pastime there — which I think is a future post). Where there was an actual problem, generally teams were already solving it their own.
I wish I had a grand conclusion, but as I said upfront, this is still an immature response, and I’m still trying to work my way through my thoughts.
