Check out the hypothetical agile burndown and EVM (Earned Value Management) charts below. Like in the “real” world, the example project (or sprint, if you prefer) ended up being underestimated. The shortfall is indicated by the dotted line on the right.
When we literally flip the agile burndown chart in the vertical dimension, we get this:
The moral of the story is: “Don’t be fooled by the agilista herd; an agile burndown chart is nothing but an inverted version of the despised EVM chart.“
If ur customer *requires* formal waterfall events like “Sys Reqs Review”, “Prelim Design Review”, “Critical Design Review”, gotta do them.
— Tony DaSilva (@Bulldozer0) March 30, 2014
The customers of all the big government-financed sensor system programs I’ve ever worked on have required the aforementioned, waterfall, dog-and-pony shows as part of their well-entrenched acquisition process. Even prior to commencing a waterfall death march, as part of the pre-win bidding process, customers also (still) require contractors to provide detailed schedule and cost commitments in their proposal submissions – right down to the CSCI level of granularity.
If you think it’s tough to get your internal executive customers to wholeheartedly embrace an “agile adoption” or “no estimates” initiative, try to wrap your mind around the cosmic difficulty of doing the same to a large, fragmented, distributed authority, external acquisition machine whose cogs are fine-tuned to: cover their ass, defend their turf, and doggedly fight to keep the extant process that justifies their worth in place. Good luck with that.
I was going through some old project stuff and stumbled upon the chart below. I developed it back when I was the software lead of a nine person sub-team on an embedded system product development effort:
Putting all those indecipherable acronyms adorning the chart aside, note that the project was performed in 2004 – a mere 3 years after the famous “Agile Manifesto” was hatched. I can’t remember if I knew about (or read) the manifesto at the time, but I do know that Tom Gilb’s “Evo“ and Barry Boehm’s “Spiral“ processes had radically changed my worldview of software development. Specifically, the (now-obvious) concept of incremental development and delivery rang my bell as the best way to mitigate risk on challenging, software-intensive, projects.
As the chart illustrates, the actual hand-off of each of the seven builds (to the system integration test team) was pretty much dead nuts right on target. Despite the fact that the project front end (requirements definition and software design) was managed as a “waterfall” endeavor, the targets were met. Thus, I’m led to believe the following trivial trivia:
Not all agile projects succeed and not all waterfall projects fail.
To BD00, the agile movement, even though it is a refreshing backlash against the “Process Models And Standards Quagmire” (PMASQ) perpetrated by a well-meaning but clueless mix of government and academic borgs who don’t have to create and build anything, has spawned its own quagmire of “Agile Process Frameworks And Practices Quagmire” (APFAPQ). Like the PMASQ community has ignited a cottage industry of expensive consultants, certifiers, assessors, trainers, and auditors, the APFAPQ movement has jump-started an equivalent community of expensive consultants, coaches, trainers, certifiers.
The figure below highlights one problem with government “governance” of big software systems development. Sure, it’s dated, but it drives home the point that there’s a standards quagmire out there, no?
Imagine that you’re a government contractor and, for every system development project you “win“, you’re required to secure “approval” from a different subset of authorities in a quagmire standards “system” like the one above. Just think of the overhead cost needed to keep abreast of, to figure out which, and to comply with, the applicable standards your product must conform to. Also think of the cost to periodically get your company and/or its products assessed and/or certified. If you ever wondered why the government pays $1000 for a toilet seat, look no further.
I look at this random, fragmented standards diagram as a paranoid, cover-your-ass strategy that government agencies can (and do) whip out when big systems programs go awry: “The reason this program is in trouble is because standards XXX and YYY were not followed“. As if meeting a set of standards guarantees robust, reliable, high-performing systems. What a waste. But hey, it’s other people’s money (yours and mine), so no problemo.
Rewritten in “old school” terminology, the five Scrum process events can be expressed as follows:
- Sprint Planning = Requirements definition and capture
- Sprint = Requirements analysis, design, coding, unit testing, integration testing, code review
- Daily Stand Up = Daily status meeting
- Sprint Review = Post-mortem
- Sprint Retrospective = Continuous process improvement
So, someone with an intentionally warped mind like BD00 may interpret a series of Scrum sprints as nothing more than a series of camouflaged Mini-Waterfalls (MW).
But ya know what? Executing a project as a series of MWs may be a good thing – as long as an arbitrary, fixed-size, time-box is not imposed on the team. After all, since everything else is allowed to dynamically change during a Scrum project, why not the size of the Sprint too?
Instead of estimating what features can be done in the next 30 days, why not simply estimate how many days will be needed to complete the set of features that marks the transition into the next MW? If, during the MW, it is learned that the goal won’t be achieved, then in addition to cancelling the MW outright, two other options can be contemplated:
- Extend the length of the MW
- Postpone the completion of one or more of the features currently being worked on
Our sophistication continuously puts us ahead of ourselves, creating things we are less and less capable of understanding – Nassim Taleb
It’s like clockwork. At some time downstream, just about every major weapons system development program runs into cost, schedule, and/or technical performance problems – often all three at once (D’oh!).
Despite what their champions espouse, agile and/or level 3+ CMMI-compliant processes are no match for these beasts. We simply don’t have the know how (yet?) to build them efficiently. The scope and complexity of these Leviathans overwhelms the puny methods used to build them. Pithy agile tenets like “no specialists“, “co-located team“, “no titles – we’re all developers” are simply non-applicable in huge, multi-org programs with hundreds of players.
Being a student of big, distributed system development, I try to learn as much about the subject as I can from books, articles, news reports, and personal experience. Thanks to Twitter mate @Riczwest, the most recent troubled weapons system program that Ive discovered is the F-35 stealth fighter jet. On one side, an independent, outside-of-the-system, evaluator concludes:
The latest report by the Pentagon’s chief weapons tester, Michael Gilmore, provides a detailed critique of the F-35′s technical challenges, and focuses heavily on what it calls the “unacceptable” performance of the plane’s software… the aircraft is proving less reliable and harder to maintain than expected, and remains vulnerable to propellant fires sparked by missile strikes.
On the other side of the fence, we have the $392 billion program’s funding steward (the Air Force) and contractor (Lockheed Martin) performing damage control via the classic “we’ve got it under control” spiel:
Of course, we recognize risks still exist in the program, but they are understood and manageable. – Air Force Lieutenant General Chris Bogdan, the Pentagon’s F-35 program chief
The challenges identified are known items and the normal discoveries found in a test program of this size and complexity. – Lockheed spokesman Michael Rein
All of the risks and challenges are understood, manageable, known? WTF! Well, at least Mr. Rein got the “normal” part right.
In spite of all the drama that takes place throughout a large system development program, many (most?) of these big ticket systems do eventually get deployed and they end up serving their users well. It simply takes way more sweat, time, and money than originally estimated to get it done.
Even though it has a title and cover design that only a Harvard MBA could love, I picked up Robert Austin’s “Measuring and Managing Performance in Organizations” on a twitter tip from Torbjörn Gyllebring. As soon as I cracked the cover, I knew it was gonna be a classic. The foreword was written by one of my all time favorite software authors, Tom DeMarco.
Mr. Austin discovered perhaps the first recorded instance of the well worn “schedule is king!” management law:
Such scenarios, in which program managers or contractors attend to measurements of timeliness of delivery to the exclusion of all else, are reported as early as 1882. In that year, the newly built U.S.S. Omaha was discovered to have onboard-coal-room for only four days’ steaming; in the rush to stay on schedule, no one had been willing to force notice of this defect at a high enough level to ensure its correction.
Every once in a blue moon, I finish a book so engrossing that I immediately reread it before cracking open a different one. Mr. Austin’s MAMPIO is one of those gems and I’m well into my second romp through it. Since it’s loaded with a gazillion ideas for blog posts, expect more over-the-top BD00 distortions to come. W00t!
The anecdotal evidence is overwhelming. Agile methods can work really well for many small teams and small projects. However, no matter what the expert, high-profile, “coaches” purport, the jury is still out regarding its scalability to large teams and large projects. In “How even agile development couldn’t keep this mega-project on track“, Nick Heath showcases the British disaster known as the £2.4bn “Universal Credit Programme“.
First, the sad fact:
…the UK government has had to write off at least £34m on the programme and delay the national launch for the project. The department in charge of the project, the Department for Work and Pensions (DWP), can’t guarantee the remainder of the £303m it has spent on the project so far will offer “good value” it said.
From the rest of Nick’s story, it becomes clear that agile methods weren’t really used to develop the software:
There was a two-year gap between the DWP starting the project design and build process, and the system going live in 2013.
The DWP experienced problems incorporating the agile approach into existing contracts, governance and assurance structures.
That second point is key. No matter how much a big org wants to be “agile“, it is heavily constrained by the hierarchical structures, stature-obsessed mindsets, byzantine processes, and form-filled procedures entrenched within not only itself, but also within its suppliers and customers. It’s a classic “system” problem where futzing around with one component may crash the whole system because of hardened interfaces and skin tight coupling.
As the figure below shows, attempting to “agilize” a large component within an even larger, waterfall-centric, system creates impedance mismatches at every interface. The greater the mismatch, the less productive the system becomes. Information flow and understanding between components bog down while noise and distortion overwhelm the communication channels. In the worse case, the system stops producing value-added output and it would have been better to leave the old, inefficient, waterfall-centric system intact.
The only chance an agile-wanna-be component has at decoupling itself from the external waterfall insanity is to covertly setup a two-faced, agile<->waterfall protocol converter for each of its external interfaces. Good luck pullin’ that stunt off.
I dunno who said it, but paraphrasing whoever did:
Science progresses as a succession of funerals.
Even though more accurate and realistic models that characterize the behavior of mass and energy are continuously being discovered, the only way the older physics models die out is when their adherents kick the bucket.
The same dictum holds true for software development methodologies. In the beginning, there was the Traditional (a.k.a waterfall) methodology and its formally codified variations (RUP, MIL-STD-498, CMMI-DEV, your org’s process, etc). Next, came the Agile fork as a revolutionary backlash against the inhumanity inherent to the traditional way of doing things.
The most recent fork in the methodology march is the cerebral SEMAT (Software Engineering Method And Theory) movement. SEMAT can be interpreted (perhaps wrongly) as a counter-revolution against the success of Agile by scorned, closet traditionalists looking to regain power from the agilistas.
On the other hand, perhaps the Agile and SEMAT camps will form an alliance and put the final nail in the coffin of the old traditional way of doing things before its adherents kick the bucket.
SEMAT co-creator Ivar Jacobson seems to think that hitching SEMAT to the Agile gravy train holds promise for better and faster software development techniques.
Who knows what the future holds? Is another, or should I say, “the next“, fork in the offing?