Even though it has a title and cover design that only a Harvard MBA could love, I picked up Robert Austin’s “Measuring and Managing Performance in Organizations” on a twitter tip from Torbjörn Gyllebring. As soon as I cracked the cover, I knew it was gonna be a classic. The foreword was written by one of my all time favorite software authors, Tom DeMarco.
Mr. Austin discovered perhaps the first recorded instance of the well worn “schedule is king!” management law:
Such scenarios, in which program managers or contractors attend to measurements of timeliness of delivery to the exclusion of all else, are reported as early as 1882. In that year, the newly built U.S.S. Omaha was discovered to have onboard-coal-room for only four days’ steaming; in the rush to stay on schedule, no one had been willing to force notice of this defect at a high enough level to ensure its correction.
Every once in a blue moon, I finish a book so engrossing that I immediately reread it before cracking open a different one. Mr. Austin’s MAMPIO is one of those gems and I’m well into my second romp through it. Since it’s loaded with a gazillion ideas for blog posts, expect more over-the-top BD00 distortions to come. W00t!
The anecdotal evidence is overwhelming. Agile methods can work really well for many small teams and small projects. However, no matter what the expert, high-profile, “coaches” purport, the jury is still out regarding its scalability to large teams and large projects. In “How even agile development couldn’t keep this mega-project on track“, Nick Heath showcases the British disaster known as the £2.4bn “Universal Credit Programme“.
First, the sad fact:
…the UK government has had to write off at least £34m on the programme and delay the national launch for the project. The department in charge of the project, the Department for Work and Pensions (DWP), can’t guarantee the remainder of the £303m it has spent on the project so far will offer “good value” it said.
From the rest of Nick’s story, it becomes clear that agile methods weren’t really used to develop the software:
There was a two-year gap between the DWP starting the project design and build process, and the system going live in 2013.
The DWP experienced problems incorporating the agile approach into existing contracts, governance and assurance structures.
That second point is key. No matter how much a big org wants to be “agile“, it is heavily constrained by the hierarchical structures, stature-obsessed mindsets, byzantine processes, and form-filled procedures entrenched within not only itself, but also within its suppliers and customers. It’s a classic “system” problem where futzing around with one component may crash the whole system because of hardened interfaces and skin tight coupling.
As the figure below shows, attempting to “agilize” a large component within an even larger, waterfall-centric, system creates impedance mismatches at every interface. The greater the mismatch, the less productive the system becomes. Information flow and understanding between components bog down while noise and distortion overwhelm the communication channels. In the worse case, the system stops producing value-added output and it would have been better to leave the old, inefficient, waterfall-centric system intact.
The only chance an agile-wanna-be component has at decoupling itself from the external waterfall insanity is to covertly setup a two-faced, agile<->waterfall protocol converter for each of its external interfaces. Good luck pullin’ that stunt off.
I dunno who said it, but paraphrasing whoever did:
Science progresses as a succession of funerals.
Even though more accurate and realistic models that characterize the behavior of mass and energy are continuously being discovered, the only way the older physics models die out is when their adherents kick the bucket.
The same dictum holds true for software development methodologies. In the beginning, there was the Traditional (a.k.a waterfall) methodology and its formally codified variations (RUP, MIL-STD-498, CMMI-DEV, your org’s process, etc). Next, came the Agile fork as a revolutionary backlash against the inhumanity inherent to the traditional way of doing things.
The most recent fork in the methodology march is the cerebral SEMAT (Software Engineering Method And Theory) movement. SEMAT can be interpreted (perhaps wrongly) as a counter-revolution against the success of Agile by scorned, closet traditionalists looking to regain power from the agilistas.
On the other hand, perhaps the Agile and SEMAT camps will form an alliance and put the final nail in the coffin of the old traditional way of doing things before its adherents kick the bucket.
SEMAT co-creator Ivar Jacobson seems to think that hitching SEMAT to the Agile gravy train holds promise for better and faster software development techniques.
Who knows what the future holds? Is another, or should I say, “the next“, fork in the offing?
When a new product development project kicks off, nobody knows squat and there’s a lot of fumbling going on before real progress starts to accrue. As the hardware and software environment is stitched into place and initial requirements/designs get fleshed out, productivity slowly but surely rises. At some point, productivity (“velocity” in agile-ese) hits a maximum and then flattens into a zero slope, team-specific, cadence for the duration. Thus, one could be led to believe that a generic team productivity/progress curve would look something like this:
In “The Year Without Pants“, Scott Berkun destroys this illusion by articulating an astute, experiential, observation:
This means that at the end of any project, you’re left with a pile of things no one wants to do and are the hardest to do (or, worse, no one is quite sure how to do them). It should never be a surprise that progress seems to slow as the finish line approaches, even if everyone is working just as hard as they were before. – Scott Berkun
Scott may have forgotten one class of thing that BD00 has experienced over his long and un-illustrious career – things that need to get done but aren’t even in the work backlog when deployment time rolls in. You know, those tasks that suddenly “pop up” out of nowhere (BD00 inappropriately calls them “WTF!” tasks).
Nevertheless, a more realistic productivity curve most likely looks like this:
If you’re continuously flummoxed by delayed deployments, then you may have just discovered why.
- The Year Without Pants: An interview with author Scott Berkun (oldienewbies.wordpress.com)
- Scott Berkun Shares Advice for Writers Working Remotely (mediabistro.com)
- A Book in 5 Minutes: “The Year without Pants: WordPress.com and the Future of Work” (tech.co)
Finally, I found out what someone actually thinks “agile practices” are. In “What are the Most Important and Adoption-Ready Agile Practices?”, Shane-Hastie presents his list:
Kudos to Shane for putting his list out there.
Ya gotta love all the “explicit definition of done” entries (“Aren’t you freakin’ done yet?“). And WTF is “Up front architecture” doing on the list? Isn’t that a no-no in agile-land? Shouldn’t it be “emergent architecture“? And no kanban board entry? What about burn down charts?
Alas, I can’t bulldozify Shane’s list too much. After all, I haven’t exposed my own agile practices list for scrutiny. If I get the itch, maybe I’ll do so. What’s on your list?
But wait! Where are the “ilities” and the other non-functional product requirements? They’re not in your backlog?
Uh, we don’t do “ilities“. It’s not agile, it’s BDUF (Big Design Up Front). We no need no stinkin’ arkitects cuz we’re all arkitects. Don’t worry. These “ilities” thingies will gloriously emerge from the future as we implement our user stories and deliver working increments every two weeks – until they stop working.
Assume (which is a bad thing to do) that you can characterize a project by three activities: Planning, Designing, and Building. Next, assume (which is still a bad thing to do) that the planning and designing activities can each be sub-characterized by “Big“, “No“, and “Just Enough“.
Forgetting about what “Just Enough” exactly means, check out this methodology-agnostic probability density function:
Do you think the curve is right? You better not. The sample size BD00 used to generate it is zero,
The figure below models the increase in team stress level versus time for waterfall and time-boxed projects. As a project nears a delivery date, the stress levels increase dramatically as the team fixes turds, integrates individually developed features into the whole, and takes care of the boring but important stuff that nobody wanted to do earlier.
One tradeoff between the two types of projects is maximum stress level vs. number of stressful events. The maximum level of experienced stress is much higher for waterfall than any one time-boxed sprint, but it only occurs once as opposed to monthly. Pick your poison: a quick death by guillotine or a slow death by a thousand cuts.
Agilistas would have everyone believe that time-boxed projects impose a constant but very low level of healthy stress on team members while waterfall quagmires impose heart-attack levels of stress on the team. They may be right, because…..
(In case you haven’t noticed, BD00 is feeling the need to use the Einstein pic above more and more in his posts. I wonder why that is.)
Lo and behold! It’s the monstrously famous iron triangle:
Even though all three critical project factors should be respected equally, BD00 put “schedule” on top because the unspoken rule is “schedule is king” in many orgs.
Everyone who’s ever worked on an important, non-boondoggle, project has heard or spoken words like these:
“I’m concerned that we’re exceeding the budget.”
“I’m afraid that we won’t meet the schedule commitment.”
But how many people have heard words like these:
“I fear that our product quality won’t meet our customer’s expectations.”
Ok, so you have heard them, but stop raining on my parade and let’s not digress.
The reason that quality concerns are mentioned so infrequently relative to cost and schedule is that the latter two objective project attributes are easily tracked by measuring the universally accepted “money” and “time” metrics. There is no single, universally accepted objective quality metric. If you don’t believe BD00, then just ask Robert Pirsig.
To raise quality up to the level of respectability that schedule and cost enjoy, BD00 proposes a new metric for measuring quality: the “WTF?“. To start using the metric, first convince all your people to not be afraid of repercussions and encourage them to blurt out “WTF?” every time they see some project aspect that violates their aesthetic sense of quality. Then, have them doggedly record the number and frequency of “WTF?”s they hear as the project progresses.
Before you know it, you’ll have a nice little histogram to gauge the quality of your project portfolio. Then you’ll be able to…, uh, you’ll be able to… do something with the data?
In Scrum, one of the three questions every team member is supposed to answer at the daily standup meeting is: “Are there any obstacles in your way?“. BD00 often wonders how many people actually answer it, let alone really answer it…
And no, BD00 doesn’t really answer it. Been there and done that. As you might surmise, because of his uncivilized nature and lack of political savvy, it didn’t work out so well – so he stopped using that industry-best practice. But for you and your team, really answering the question works wonders – right?
The day soldiers stop bringing you their problems is the day you have stopped leading them. They have either lost confidence that you can help or concluded that you don’t care. Either case is a failure of leadership.” – Karl Popper