Refactoring versus Rewriting
I started my first real Agile software development project in 1999. I'd been doing more traditional software development before then all the way back to 1980. I won't bore you with the details of those earlier projects, but my feeling was that there had to be a better way of developing software that didn't involve a senior technologist (me) telling a whole bunch of junior technologists what to do. It turns out I was right. 🙂
But almost from the start I got pushback from other people in the development organizations I worked in that Agile development was horribly wasteful. They pointed to Test Driven Development ("all those tests more than double your effort"), pair programming ("two developers doing the work of one?"), and refactoring ("you're rewriting the software every time at enormous cost"). Of course all of these objections were born not just out of a misunderstanding of Agile development, but a fundamental misunderstanding of how their own software development processes actually worked.
The issue of refactoring was particularly mysterious to my colleagues. If you took the time and designed software properly up front, they reasoned, you won't have to do expensive rewrites. Of course anyone who has ever maintained code knows that this isn't ever true. All code, regardless of how it is developed, changes over time for bug fixes and to meet the needs of new requirements. If you aren't thoughtful about how you change your software, you can easily end up with a big mess. Further, very little software is design with a perfect understanding of the requirements, leading to imperfect designs and things like the second and third system effects.
If you are thoughtful about how you change your software and think about the sorts of "code smells" that crop up in your code over time, you end up making more fundamental changes, rather than adding a method here or a instance variable there. You might see that two methods are always called in conjunction and decide they should be folded into one method (a design win, see State Patterns), or you may have extended your design and ended up with two parallel hierarchies of classes. You break the Go4 glass and pull out the Bridge Pattern to solve that burgeoning design problem. Through constant refactoring (or rewriting, take your pick) you avoid painting yourself into a corner as your requirements change.
The fact is that good developers are constantly redesigning and rewriting their code. Those same developers will tell you that the longer you leave problems (technical debt) before refactoring, the bigger and more expensive they become, until you might be better off just scrapping the system and rewriting from scratch.
Agile development takes this normal refactoring to it's logical extreme. Rather than doing a big design up front and then doing a series of expensive changes when they are inevitably found to be wrong, Agile teams accept that their design will be imperfect and then build comparatively inexpensive refactoring into every iteration. It may seem counterintuitive, but all that refactoring results in code that is cheaper to develop, easier to maintain (because it's easier to understand, change and debug).
Among professional writers the golden rule is that the key to all good writing is rewriting. The same is certainly true of software.