Check out this slide I plucked from a pitch that will remain unnamed:
Notice the note under the waterfall diagram. Now, let’s look at the original, “unadapted” version and accompanying quote from Winston W. Royce’s classic 1970 paper:
Notice that Mr. Royce clearly noted in his paper that the sequential, never-look-back, waterfall process is a stone cold loser. Next, let’s look at another diagram from Mr. Royce’s paper; one that no fragilista ever mentions or shows:
OMG! An iterative waterfall with feedback loops? WTF!
Finally, let’s look at BD00′s syntegrated version of the agile, lower half of our consultant’s diagram and the iterative waterfall diagram from Mr. Royce’s paper:
Comparing the agile and “chunked“, iterative, waterfall models shows that, taken in the right context, they’re not that different…. no?
Rewritten in “old school” terminology, the five Scrum process events can be expressed as follows:
- Sprint Planning = Requirements definition and capture
- Sprint = Requirements analysis, design, coding, unit testing, integration testing, code review
- Daily Stand Up = Daily status meeting
- Sprint Review = Post-mortem
- Sprint Retrospective = Continuous process improvement
So, someone with an intentionally warped mind like BD00 may interpret a series of Scrum sprints as nothing more than a series of camouflaged Mini-Waterfalls (MW).
But ya know what? Executing a project as a series of MWs may be a good thing – as long as an arbitrary, fixed-size, time-box is not imposed on the team. After all, since everything else is allowed to dynamically change during a Scrum project, why not the size of the Sprint too?
Instead of estimating what features can be done in the next 30 days, why not simply estimate how many days will be needed to complete the set of features that marks the transition into the next MW? If, during the MW, it is learned that the goal won’t be achieved, then in addition to cancelling the MW outright, two other options can be contemplated:
- Extend the length of the MW
- Postpone the completion of one or more of the features currently being worked on
I’m not a fan of “emergent global architecture“, but I AM a fan of “emergent local design“. To mitigate downstream technical and financial risk, I believe that one has to generate and formally document an architecture at a high level of abstraction before starting to write code. To do otherwise would be irresponsible.
The figure below shows a portion of an initial “local” design that I plucked out of a more “global” architectural design. When I started coding and unit testing the cluster of classes in the snippet, I “discovered” that the structure wasn’t going work out. The API of the architectural framework within which the class cluster runs wouldn’t allow it to work without some major, internal, restructuring and retesting of the framework itself.
After wrestling with the dilemma for a bit, the following workable local design emerged out of the learning acquired via several wretched attempts to make the original design work. Of course, I had to throw away a bunch of previously written skeletal product and test code, but that’s life. Now I’m back on track and moving forward again. W00t!
Since Martin Fowler works there, I thought ThoughtWorks Inc. must be great. However, after watching two of his fellow ThoughtWorkers give a talk titled “From Agility To Anti-Fragility“, I’m having second thoughts. The video was a relatively lame attempt to jam-fit Nassim Taleb’s authentic ideas on anti-fragility into the software development process. Expectedly, near the end of the talk the presenters introduced their “new” process for making your borg anti-fragile: “Continuous Delivery/Discovery/Design“. Lookie here, it even has a superscript in its title:
Having read Mr. Taleb’s four fascinating books, the one hour and twenty-six minute talk was essentially a synopsis of his latest book, “Anti-Fragile“. That was the good part. The ThoughtWorkers’ attempts to concoct techniques that supposedly add anti-fragility to the software development process introduced nothing new. They simply interlaced a few crummy slides with well-known agile practices (small teams, no specialists, short increments, co-located teams, etc) with the good slides explaining optionality, black/grey swans, convexity vs concavity, hormesis, and levels of randomness.
Having recently watched a newer incarnation of Barbara Liskov‘s terrific Turing award acceptance speech on InfoQ.com, “The Power Of Abstraction“, I started doodling on my visio canvas to see where it would take me. Somehow, I wanted to explore how the use of abstraction imbues power to its wielders.
The figure below attempts to represent 3 different software designs that can result from the analysis of a given set of requirements (how the requirements came to be “given” in the first place is a whole ‘nother issue).
On the left, we have a seven class solution candidate (C1….. C7 ) organized as three layers of abstraction. On the right, we have a three class flat solution (FC1, FC2, FC3) that implements the same functionality (e.g. FC1 encapsulates the functionality of C1 + C4 + C7). For dramatic contrast, we have a fugly, single-class, monolith in the middle with all the solution functionality entombed within the MC1 class sarcophagus.
So, what advantage, if any, does the three tier, abstract design give stakeholders over the two, flat, down-to-earth designs? Depending on the requirements specifics, it may offer up no advantage and might actually be the worst candidate in terms of code-ability, understandability, and maintainability. There are more “parts” and more inter-part interfaces. It may be overkill to transform the requirements into 3 layers of abstraction before (or during?) coding.
However, as a system to be coded gets larger and more complex, the intelligent use of abstract vertical layering and horizontally balancing can speed up system development and decrease maintenance costs via increased readability and understandability from multiple viewing angles. For large systems, conceptual “chunking“, both vertically in the form of layering and horizontally in the form of balancing is a winning strategy; especially when coupled with Miller’s magic number 7 (no more than 7 +/- 2 abstract elements within a given layer and no more than 7 +/- 2 abstract layers in the stack). Relatively speaking, the smaller, bounded parts can be doled out to team members more easily and integration will be less painful.
Note that doing some just-enough “pre-planning” in terms of layering/balancing the system’s structure/behavior seems to fly in the face of TDD – where you sprinkle a bunch of user stories from the backlog onto a group of programmers and have them start writing tests so that the design can miraculously emerge. But, as the saying goes: “whatever floats your boat“.
When I started this blog four years ago, I had to decide whether to publish as an anonymous coward or to use my real name. I struggled with the decision for a bit because I knew I was going to write frequently, real frequently, about dysfunctional management and institutional behaviors that I’ve both experienced and (even more so) read about over the years. In addition, since I’m a high energy, passionate animal who doesn’t hold much back and at times finds it hard to compromise, I knew that much of my content was going be highly caustic and offensive.
Out of fear of repercussions, I decided to start writing incognito… until a dear friend brought up the perplexing issue again. After rethinking the situation, I resolved to let it all hang out. I gingerly hoisted my name up on my “About me” page. Never say never, but I didn’t (and still don’t) care about climbing any corpo ladder or presenting the squeaky clean image that all main stream “leadership” books tout as necessary to “get ahead“. I have some hairy warts and barnacles growing on my brain and, hell, I choose to expose them.
So, if I don’t want to get ahead by movin’ on up, then WTF does BD00 want? I want to keep ruining drill bits while I blast away at the impenetrable bedrock that entombs the holy grail of effective software development. I like going deep, deep, deep down into the unexplored corners of programming (in C++, of course), design, architecture, requirements, and the squishy realm of team-based software development processes. These closely-coupled topics excite me because there seems to be no bottom, no final “truths“, no end to life-long learning in any of them. It’s what I was meant to do.
What were you meant to do?
When not ranting and raving on this blawg about “great injustices” (LOL) that I perceive are keeping the world from becoming a better place, I design, write, and test real-time radar system software for a living. I use the UML before, during, and after coding to capture, expose, and reason about my software designs. The UML artifacts I concoct serve as a high level coding road map for me; and a communication tool for subject matter experts (in my case, radar system engineers) who don’t know how to (or care to) read C++ code but are keenly interested in how I map their domain-specific requirements/designs into an implementable software design.
I’m not a UML language lawyer and I never intend to be one. Luckily, I’m not forced to use a formal UML-centric tool to generate/evolve my “bent” UML designs (see what I mean by “bent” UML here: Bend It Like Fowler). I simply use MSFT Visio to freely splat symbols and connections on an e-canvas in any way I see fit. Thus, I’m unencumbered by a nanny tool telling me I’m syntactically/semantically “wrong!” and rudely interrupting my thought flow every five minutes.
The 2nd graphic below illustrates an example of one of my typical class diagrams. It models a small, logically cohesive cluster of cooperating classes that represent the “transmit timeline” functionality embedded within a larger “scheduler” component. The scheduler component itself is embedded within yet another, larger scale component composed of a complex amalgam of cooperating hardware and software components; the radar itself.
When fully developed and tested, the radar will be fielded within a hostile environment where it will (hopefully) perform its noble mission of detecting and tracking aircraft in the midst of random noise, unwanted clutter reflections, cleverly uncooperative “enemy” pilots, and atmospheric attenuation/distortion. But I digress, so let me get back to the original intent of this post, which I think has something to do with how and why I use the UML.
The radar transmit timeline is where other necessarily closely coupled scheduler sub-components add/insert commands that tell the radar hardware what to do and when to do it; sometime in the future relative to “now“. As the radar rotates and fires its sophisticated, radio frequency pulse trains out into the ether looking for targets, the scheduler is always “thinking” a few steps ahead of where the antenna beam is currently pointing. The scheduler relentlessly fills the TxTimeline in real time with beam-specific commands. It issues those commands to the hardware early enough for the hardware to be able to queue, setup, and execute the minute transmit details when the antenna arrives at the desired command point. Geeze! I’m digressing yet again off the UML path, so lemme try once more to get back to what I originally wanted to ramble about.
Being an unapologetic UML bender, and not a fan of analysis-paralysis, I never attempt to meticulously show every class attribute, operation, or association on a design diagram. I weave in non-UML symbology as I see fit and I show only those elements I deem important for creating a shared understanding between myself and other interested parties. After all, some low level attributes/operations/classes/associations will “go away” as my learning unfolds and others will “emerge” during coding anyway, so why waste the time?
Notice the “revision number” in the lower right hand corner of the above class diagram. It hints that I continuously keep the diagram in sync with the code as I write it. In fact, I keep the applicable diagram(s) open right next to my code editor as I hack away. As a PAYGO practitioner, I bounce back and forth between code & UML artifacts whenever I want to.
The UML sequence diagram below depicts a visualization of the participatory role of the TxTimeline object in a larger system context comprised of other peer objects within the scheduler. For fear of unethically disclosing intellectual property, I’m not gonna walk through a textual explanation of the operational behavior of the scheduler component as “a whole“. The purpose of presenting the sequence diagram is simply to show you a real case example that “one diagram is not enough” for me to capture the design of any software component containing a substantial amount of “essential complexity“. As a matter of fact, at this current moment in time, I have generated a set of 7+ leveled and balanced class/sequence/activity diagrams to steer my coding effort. I always start coding/testing with class skeletons and I iteratively add muscles/tendons/ligaments/organs to the Frankensteinian beast over time.
In this post, I opened up my trench coat and
showed you my… attempted to share with you an intimate glimpse into the way I personally design & develop software. In my process, the design is not done “all upfront“, but a purely subjective mix of mostly high and low level details is indeed created upfront. I think of it as “Big Design, But Not All Upfront“.
Despite what some code-centric, design-agnostic, software development processes advocate, in my mind, it’s not just about the code. The code is simply the lowest level, most concrete, model of the solution. The practices of design generation/capture and code slinging/testing in my world are intimately and inextricably coupled. I’m not smart enough to go directly to code from a user story, a one-liner work backlog entry, a whiteboard doodle, or a set of casual, undocumented, face-to-face conversations. In my domain, real-time surveillance radar systems, expressing and capturing a fair amount of formal detail is (rightly) required up front. So, screw you to any and all NoUML, no-documentation, jihadists who happen to stumble upon this post.
I dunno who said it, but paraphrasing whoever did:
Science progresses as a succession of funerals.
Even though more accurate and realistic models that characterize the behavior of mass and energy are continuously being discovered, the only way the older physics models die out is when their adherents kick the bucket.
The same dictum holds true for software development methodologies. In the beginning, there was the Traditional (a.k.a waterfall) methodology and its formally codified variations (RUP, MIL-STD-498, CMMI-DEV, your org’s process, etc). Next, came the Agile fork as a revolutionary backlash against the inhumanity inherent to the traditional way of doing things.
The most recent fork in the methodology march is the cerebral SEMAT (Software Engineering Method And Theory) movement. SEMAT can be interpreted (perhaps wrongly) as a counter-revolution against the success of Agile by scorned, closet traditionalists looking to regain power from the agilistas.
On the other hand, perhaps the Agile and SEMAT camps will form an alliance and put the final nail in the coffin of the old traditional way of doing things before its adherents kick the bucket.
SEMAT co-creator Ivar Jacobson seems to think that hitching SEMAT to the Agile gravy train holds promise for better and faster software development techniques.
Who knows what the future holds? Is another, or should I say, “the next“, fork in the offing?
When a new product development project kicks off, nobody knows squat and there’s a lot of fumbling going on before real progress starts to accrue. As the hardware and software environment is stitched into place and initial requirements/designs get fleshed out, productivity slowly but surely rises. At some point, productivity (“velocity” in agile-ese) hits a maximum and then flattens into a zero slope, team-specific, cadence for the duration. Thus, one could be led to believe that a generic team productivity/progress curve would look something like this:
In “The Year Without Pants“, Scott Berkun destroys this illusion by articulating an astute, experiential, observation:
This means that at the end of any project, you’re left with a pile of things no one wants to do and are the hardest to do (or, worse, no one is quite sure how to do them). It should never be a surprise that progress seems to slow as the finish line approaches, even if everyone is working just as hard as they were before. – Scott Berkun
Scott may have forgotten one class of thing that BD00 has experienced over his long and un-illustrious career – things that need to get done but aren’t even in the work backlog when deployment time rolls in. You know, those tasks that suddenly “pop up” out of nowhere (BD00 inappropriately calls them “WTF!” tasks).
Nevertheless, a more realistic productivity curve most likely looks like this:
If you’re continuously flummoxed by delayed deployments, then you may have just discovered why.
- The Year Without Pants: An interview with author Scott Berkun (oldienewbies.wordpress.com)
- Scott Berkun Shares Advice for Writers Working Remotely (mediabistro.com)
- A Book in 5 Minutes: “The Year without Pants: WordPress.com and the Future of Work” (tech.co)