Take a look at these three state machine models for intimately developing a chunk of functionally cohesive software:
The key distinguishing feature of the two machines on the right from the pure TDD machine on the left is that some level of design is the initial driver, informer, of the subsequent coding/testing development process. Note that all three methods contain feedback transitions triggered by “learning as we go” events.
In my understanding of TDD, no time is “wasted” upfront thinking about, or capturing, design data at any level of granularity. The pithy mandate from the TDD gods is “red-green-refactor” or die. The design bubbles up solely from the testing/coding cycle in a zen-like flow of intelligence.
Personally, I work in accordance with the DDT model. How about you? For newbies who were solely taught, and only know how to do, TDD, have you ever thought about trying the “traditional” DDT way?
BTW, I learned, tried, and then rejected, TDD as my personal process from “Unit Test Frameworks“. Except for the parts on TDD, it’s a terrific book for learning about unit testing.
Since “design” is an intimately personal process, whatever works for you is fine by BD00. But just because it’s “newer” and has a lot of rabid fan-boys promoting it (including some big and famous consultants), don’t auto-assume TDD is da bomb.
Design is an intimate act of communication between the creator and the created. – Unknown
It seems to have taken awhile, but people are finally speaking out strongly against the papal infallibility of the TDD high priesthood. Michał Bartyzel, Andrew Binstock, and Jim Coplien (who actually has been speaking out against it for years) are a few of these blasphemous heretics.
Any of the “driven” techniques can work, but to insinuate that they are the “only way(s)” to build superior products is arrogant, hubristic, and plain stupid. In mentally challenging, knowledge-intensive work, people have, and always will have, different ways of creating beautiful products.
When I speak with adherents of test-driven development (TDD) in particular, there is a seeming non-comprehension that truly excellent, reliable code was ever developed prior to the advent of this one practice. I sense their view that the long history of code that put man on the moon, ran phone switches, airline reservation systems, and electric grids was all the result of luck or unique talents, rather than a function of careful discipline and development rigor. – Andrew Binstock
“…these promises were not supported by unambiguous and verifiable data in early years of TDD. The enthusiastic reaction to TDD came first and then some (selective?) measurements were made to verify its promises. – Michał Bartyzel
“If you find your testers (or yourself) splitting up functions to support the testing process, you’re destroying your system architecture and code comprehension along with it. Test at a coarser level of granularity.” – Jim Coplien
Personally, I start sketching out quite a bit of a design upfront (OMG!!!!!) in “bent” UML (OMG, OMG!!!!!) prior to writing a single line of code. I then start writing the code and I subsequently write/run some selective unit tests on that code. In summary, I dynamically build/test the code base toward the coarse, “upfront” design as I go. Of course, according to the high priests of TDD, I’m unprofessional and I obviously produce inferior designs and bug-ridden, unmaintainable code. Gosh, it sux to be me.
For more details on how I develop software, check out this four year old post: PAYGO II. And no, I’m not promoting it as better than your personal process or, gasp, the unassailable holy grail of software development… TDD! No books, magazine articles, conference talks, or two-day certification courses are planned. It’s simply better for me, and perhaps only me.
I get e-mails like this all the time:
In four fun-filled days, and for a measly $2000, you too can become a full fledged, CERTIFIED Scrum Master AND Scrum Product Owner. That’s all it takes to catapult you to the top of the software development world. W00t!
Why spend weeks or months struggling to learn a new language, API, framework, technology, or tool when you can nail this trifecta: leapfrog over your programmer peers, break into the highly coveted guild of management, and prefix a big fat “C” to your title? Quick, quick, jump on the bandwagon before it’s too late. Don’t get left behind!
“A person is smart, people are stupid.” – Agent K (Men in Black)
I’m forever fascinated by how large groups of bright and well-meaning individuals often behave so dysfunctionally as a “whole“. How can that be? It’s because the individual “parts” of an organization don’t determine the behavior of the whole. It’s the part-to-part interactions enabled by, and (more importantly) disabled by, the structure of an organization that determines system behavior. By default, hierarchically structured orgs suppress collaborative communications in the horizontal dimension while catalyzing top down command and control communications in the vertical dimension.
In the vertical dimension, the fact that bosses get to unconditionally decide on how to divvy out tasks and rewards to their “subordinates” below ensures that the dweebs in the lower tiers will do whatever the boss wants them to do with little, if any, frictional blowback. In the horizontal dimension, crucial information can be withheld and collaborative communication suppressed because of peer-to-peer competition for the limited number of coveted slots available upstairs in the ever narrowing pyramid.
When it comes down to it, the primary function of management is to allocate finite org resources to efforts that will subsequently increase revenues and/or decrease costs. By resources, I mean people and money (for salaries, materials, tools, training, etc) and time.
Since projects vary wildly in complexity/importance/size, required skill-sets, and they (should) have end points, correctly allocating resources to, and amongst, projects is a huge challenge. Both over and under allocation of resources can threaten the financial viability of the org and, thus, everyone within it. Under-allocation can lead to a stagnant product/service portfolio and over-allocation can lead to an expensive product/service portfolio. Note that either under or over allocation can produce individual project failures.
To correctly allocate resources to projects, especially the “human resources“, some things must be semi-known about them. Besides desired outcomes and required execution skills, project parameters like size and/or complexity should be estimated to some level of granularity. That’s why, for the most part, I think the #noestimates and #noprojects people are full of bullocks. Like agile coaches, they mean well and their Utopian ideas sound enticing. Thanks, but no thanks.
Fresh from Tom Gilb’s “Advanced Agile Practices” presentation, I give you Dave Rico’s 14 pitfalls of agile methods:
If you look closely at the list, the entries don’t just apply to attempts at agilization. They are daunting challenges for any aspiring corpo change agent who wishes to make a sweeping change to “the way we develop products“.
The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise. — Edsger Dijkstra
With Edsger’s delicious quote in mind, let’s explore seven levels of abstraction that can be used to reason about big, distributed, systems:
At level zero, we have the finest grained, most concrete unit of design, a single puny line of “source code“. At level seven, we have the coarsest grained, most abstract unit of design, the mysterious and scary “system” level. A line of code is simple to reason about, but a “system” is not. Just when you think you understand what a system does, BAM! It exhibits some weird, perhaps dangerous, behavior that is counter-intuitive and totally unexpected – especially when humans are the key processing “nodes” in the beast.
Here are some questions to ponder regarding the seven level stack: Given that you’re hired to build a big, distributed system, at what level would you start your development effort? Would you start immediately coding up classes using the much revered TDD “best practice” and let all the upper levels of abstraction serendipitously “emerge”? Relatively speaking, how much time “up front” should you spend specifying, designing, recording, communicating the structures and behaviors of the top 3 levels of the stack? Again, relatively speaking, how much time should be allocated to the unit, integration, functional, and system levels of testing?
If ur customer *requires* formal waterfall events like “Sys Reqs Review”, “Prelim Design Review”, “Critical Design Review”, gotta do them.
— Tony DaSilva (@Bulldozer0) March 30, 2014
The customers of all the big government-financed sensor system programs I’ve ever worked on have required the aforementioned, waterfall, dog-and-pony shows as part of their well-entrenched acquisition process. Even prior to commencing a waterfall death march, as part of the pre-win bidding process, customers also (still) require contractors to provide detailed schedule and cost commitments in their proposal submissions – right down to the CSCI level of granularity.
If you think it’s tough to get your internal executive customers to wholeheartedly embrace an “agile adoption” or “no estimates” initiative, try to wrap your mind around the cosmic difficulty of doing the same to a large, fragmented, distributed authority, external acquisition machine whose cogs are fine-tuned to: cover their ass, defend their turf, and doggedly fight to keep the extant process that justifies their worth in place. Good luck with that.
I was going through some old project stuff and stumbled upon the chart below. I developed it back when I was the software lead of a nine person sub-team on an embedded system product development effort:
Putting all those indecipherable acronyms adorning the chart aside, note that the project was performed in 2004 – a mere 3 years after the famous “Agile Manifesto” was hatched. I can’t remember if I knew about (or read) the manifesto at the time, but I do know that Tom Gilb’s “Evo“ and Barry Boehm’s “Spiral“ processes had radically changed my worldview of software development. Specifically, the (now-obvious) concept of incremental development and delivery rang my bell as the best way to mitigate risk on challenging, software-intensive, projects.
As the chart illustrates, the actual hand-off of each of the seven builds (to the system integration test team) was pretty much dead nuts right on target. Despite the fact that the project front end (requirements definition and software design) was managed as a “waterfall” endeavor, the targets were met. Thus, I’m led to believe the following trivial trivia:
Not all agile projects succeed and not all waterfall projects fail.