Take a minute to look around yourself and imagine how many interdependent chains of specialists and handoffs were required to produce the materials that make your life quite a bit more comfortable than our cave dwelling ancestors.
Total self sufficiency is a great concept, but if you had to do all the work to produce all of the things you take for granted, you wouldn’t be able to. You simply wouldn’t have enough time.
In software development, above a certain (unknown) level of complexity and size, specialists, dependencies, and handoffs are required to have any chance of getting the work done and the product integrated. Cross-functional teams, where every single person on the team knows how to design and code every functional area of a product, is an unrealistic pipe dream for anything but trivial products.
On my first pass through Bertrand Meyer’s “Agile!” book, I interpreted its contents as a well-reasoned diatribe against agilism (yes, “agile” has become so big that it warrants being called an “ism” now). Because I was eager to do so, I ignored virtually all the positive points Mr. Meyer made and fully embraced his negative points. My initial reading was a classic case of the “confirmation bias“; where one absorbs all the evidence aligning with one’s existing beliefs and jettisons any evidence to the contrary. I knew that the confirmation bias would kick in before I started reading the book – and I looked forward to scratching that ego-inflating itch.
On my second pass through the book, I purposely skimmed over the negatives and concentrated on the positives. Here is Mr. Meyer’s list of positive ideas and practices that “agile” has either contributed to, or (mostly) re-prioritized for, the software development industry:
- The central role of production code over all else
- Tests and regression test suites as first class citizens
- Short, time-boxed iterations
- Daily standup meetings
- The importance of accommodating change
- Continuous integration
- Velocity tracking & task boards
I should probably end this post here and thank the agile movement for refocusing the software development industry on what’s really important… but I won’t :)
The best and edgiest writing in the book, which crisply separates it from its toned down (but still very good) peer, Boehm and Turner’s “Balancing Agility And Discipline“, is the way Mr. Meyer gives the agilencia a dose of its own medicine. Much like some of the brightest agile luminaries (e.g. Sutherland, Cohn, Beck, Jeffries, Larman, Cockburn, Poppendieck, Derby, Denning (sic)) relish villainizing any and all traditional technical and management practices in use before the rise of agilism, Mr. Meyer convincingly points out the considerable downsides of some of agile’s most cherished ideas:
- User stories as the sole means of capturing requirements (too fine grained; miss the forest for the trees)
- Pair programming and open offices (ignores individual preferences, needs, personalities)
- Rejection of all upfront requirements and design activities (for complex systems, can lead to brittle, inextensible, dead-end products)
- Feature-based development and ignorance of (inter-feature) dependencies (see previous bullet)
- Test Driven Development (myopic, sequential test-by-test focus can lead to painting oneself into a corner).
- Coach as a separate role (A ploy to accommodate the burgeoning agile consulting industry. Need more doer roles, not talkers.)
- Embedded customer (There is no one, single, customer voice on non-trivial projects. It’s impractical and naive to think there is one.)
- Deprecation of documents (no structured repository of shared understanding to go to seek clarification on system level requirements/architecture/design issues; high maintenance costs for long-lived systems; costly on-boarding of new developers)
I’ve always maintained that there is much to like about the various agile approaches, but the way the agile big-wigs have been morphing the movement into a binary “do-this/don’t-do-that” religious dogma, and trashing anything not considered “agile“, is a huge turnoff to me. How about you?
When creating an architecture or detailed design of a system component, up to three forms of design evaluation can be performed:
- A: Continuous evaluation by the designer during the process
- B: Evaluation by a peer group at one or more review events during the process
- C: Evaluation by an expert outsider group after A and B
These three forms of evaluation are illustrated in the sequence of diagrams below. Note that as each evaluation type is progressively added to the mix, a new feedback loop is introduced into the system.
During A, the conceive/evaluate/repair loop is performed at the speed of thought within the isolated mind of the designer. As the designer applies his skills to his understanding of the problem to be solved, alternative component structures and configurations can pop up instantaneously during each of many high frequency loop traversals. At some point, the designer concludes that his design solution is stable enough for external scrutiny and/or ready for the next step of realization.
During B, the designer walks through his creation in front of a group of peers. He explains the structure and behavior mechanisms of his design and how it solves the problem. If the peer group is qualified, prepared, and objective (QPO), at least a portion of the feedback the group produces will be valuable to the designer and he will use it to improve his design. However, unless all three QPO pre-conditions are satisfied, the peer review process will be a huge waste of time and money.
Jetting in an expert outsider group is the riskiest type of evaluation. Assuming that the QPO criteria is satisfied, the fact that the expert outsider group has no real “skin in the game” after it vacates the premises should be a cause for concern to anyone contemplating the use of the technique.
In general, the more negative feedback loops incorporated into a process, the more likely the process is to produce its intended output. As the final figure above implies, incorporating all three types of evaluation into a design review process can lead to a high quality design. However, blindly ignoring the QPO criteria and/or failing to address the “skin in the game” risk can increase cost, lengthen schedule, and lay waste to your well-intentioned review process – without you ever knowing that it occurred. As ever, the devil is in the details.
Either directly or subliminally, the message I hear from hardcore agilista big-wigs is that an agile process trumps a traditional plan-driven software development process every time and in every context – no exceptions.
On the other hand, the message I hear from traditionalists is… well, uh, I don’t hear much from traditionalists anymore because they’ve been beaten into silence by the hordes of unthinking zombies unleashed upon them by the agilista overlords.
Regarding the “betterness” of #agile over #traditional (or #noestimates over #estimates, or #noprojects over #projects), please leave your handful of personal anecdotes at home. Charismatic “I’ve seen” and “in my (vast) experience” stories don’t comprise science and aren’t sufficient justifications for sweeping generalizations. The science simply doesn’t exist – especially for the construction of large, distributed software systems.
I suppose that if a plausible (and thus, falsifiable) theory of software development processes was to be methodically derived from first principles and rigorously tested via a series of repeatable experiments, the general result would end up looking something like this:
What predictive capabilities do you think a credible theory of software development processes would generate? Show Me Your Curves.
Wanna go on a wildly fun rollercoaster ride? Then watch Erik Meijer’s “One Hacker Way” rant. Right out of the gate, I guesstimate that he alienated at least half of his audience with his opening “if you haven’t checked in code in the last week, what are you doing at a developer’s conference?” question.
I didn’t agree with all of what Erik said (I doubt anyone did), but I give him full credit for sticking his neck out and attacking as many sacred cows as he could: Agile, Scrum, 2 day certification, TDD “waste“, non-hackers, planning poker, the myth of self-organizing teams, etc. Mmmm, sacred cows make the best tasting hamburgers.
Uncle Bob Martin, the self-smug pope who arrogantly proclaimed “If you don’t do TDD, you’re unprofessional“, tried to make light of Erik’s creative rant with this lame blog post: “One Hacker Way!“. Nice try Bob, but we know you’re seething inside because many of your sacredly held beliefs were put on the stand. You seem to enjoy hacking the sacred cows of the reviled “traditional” way of developing software, but it’s a different story when your own cutlets are at steak.
Mr. Meijer pointed out what I, and no doubt many others, have thought for years: agile, particularly Scrum, is a subtle, insidious form of control. At least with explicit, transparent control, one knows the situation and where one stands. With Scrum, the koolaid-guzzling flock is duped into believing they’re in control; all the while being micro-managed with daily standups, burndown charts, velocity tracking, and cutesy terminology. No wonder it’s amassed huge fame and success – managers love Scrum because it makes their job easier and anesthetizes the coders.
My fave laugh-out-loud moment in Erik’s talk was when he presented Jeff Sutherland as the perceived messiah of software development:
I’ve found that the best books and talks are those in which I find some of the ideas enlightening and some revolting. Erik Meijer’s talk is certainly one of those brain-busting gems.
The test of a first-rate intelligence is the ability to hold two opposing ideas in mind at the same time and still retain the ability to function. – F. Scott Fitzgerald
I’ve been developing software in the aerospace and defense industry for 30+ years. Because of the sizes of, and innate System-of-Systems nature of, the solutions in this domain, the processes used to develop them are by necessity, heavyweight. Oh sure, agile/lean techniques can be used effectively in the teeny-tiny-small deep down in the bowels of some of these programs, but to assume “scaling agile” will work in a domain with hundreds of programmers from multiple contractors working on the same distributed system is naive at best, and downright disingenuous at worst.
The most ridiculous examples of inapplicable development practices to the aerospace and defense industry are #noestimates” and “#noprojects“. The “#noprojects” cult is so far off base that they don’t even acknowledge the existence of “programs” – which are a way of organizing a large set of inter-dependent “#noprojects” into, well, a larger grained mission-critical entity worth tens of millions of dollars. It’s like before there were classes and namespaces(packages) added to a programming language to accommodate increasing software system size, there were only “functions” for organizing the structure of a program – and having functions as your only organizing mechanism doesn’t scale well to large software systems.
Hell, the #noprojects people are akin to #noclasses and #nonamespaces people, of which there are thankfully, #nomembers. Really hardcore #noprojects people are perhaps even more loony. They are equivalent to a hypothetical #nofunctionsallowed crowd that demands monolithic, straight-line code only. Line number 1 straight to line number 5,000,000 – and you can’t use loops or “ifs” because they don’t add value and #nofunctionsallowed is all about “maximizing the amount of work not done“.
But hey, if you don’t need any “projects” to build a successful product from the gecko, then by extrapolation you don’t need any “programs” for organizing your projects. Hence, the inimitable BD00 proposes to “scale up” the “#noprojects” movement. We’ll start a revolutionary #noprograms movement, or equivalently, #justcode . We’ll leapfrog both the #noestimates and #noprojects movements at once. W00t!
At its core, process agility is all about continuous learning, fast feedback loops, and fluid changeability. Unlike pre-agile methods (and even some currently purported agile methods), which assume that people are forward-marching automatons who “better not make mistakes” and must defend the fort against all external forces of change, process agility accommodates the mental limitations and fallibility of REAL human beings.
Having said that, how agile do you think a process which includes a sign-off list like this is:
Imagine that whatever has been “approved” by a ceremonial sheet like this is post-facto found to be laced with errors, inconsistencies, and ambiguities due to natural human fallibility. How likely do you think that finders-of-mistakes will publicly point them out, demand a production line stoppage to fix the turds, and suggest that the director-manager-lead approval gauntlet be traversed again? Conversely, how likely do you think that finders-of-mistakes will say “f*ck-it!“, keep their mouths shut, and keep goose-stepping forward with the herd.
Fear not, dear reader. BD00 has a simple and clean solution to the director-manager-lead approval gauntlet problem. Collapse the list of approvers down to one – the only one that matters:
Please submit your plans for BD00 approval in the comments section. As his executive assistant, I can assure you that his stamp/no-stamp decision will be made pronto. However, don’t call us. We’ll call you.