Why Adopt Agile?

On a recent discussion board, a fellow LinkedIn member asked the question,
"Why Adopt Agile?" 

Here's his complete question:

"Hey all,
I figure we often get asked the question by executives and engineering teams "Why should we adopt agile?" I wanted to better articulate my argument to my team, and hope that a few of you may bring insight into reasons to adopt Agile that I haven't thought of before."
http://www.linkedin.com/groups/Why-Adopt-Agile-81065.S.5842362499172704258?view=&srchtype=discussedNews&gid=81065&item=5842362499172704258&type=member&trk=eml-anet_dig-b_pd-ttl-cn&fromEmail=&ut=1XBI14jvlLxm81

Since I believe he is probably not the only one with this question, I have decided to share my response here:

Communication.  Communication. Communication.  Everything we do boils down to communicating. In Software development we get requirements from the business and attempt to turn them into software.  In the waterfall world you gather all the requirements in front with the assumption that all the requirements can be gathered and they won't change over time.  If you can get a comprehensive list of requirements and spend enough time up front figuring out how you will build it, if the business approves the design, you'll be set.  The problem is those assumptions are simply wrong.  Requirements will change. The team’s understanding of requirements will change. Problems that could not be foreseen in development will arise and they will impact the design.  These are the main reasons Waterfall projects fail.  Sure they try to manage things like scope and make it very hard to change scope. But if you don't allow the scope to change, you will deliver a product that is no longer what the business wants or needs.  On the other hand we all know that allowing scope to change in a world of Big Up Front Design (BUFD) is extremely expensive.

Agile addresses these issues through effective and frequent communication with the product owners (the business). Short Sprints give short feedback loops.  Constant discussions about changing priorities and requirements can be addressed, entered into a backlog, and tackled with very low cost in the very next Sprint.  Requirements are gathered via user stories.  The reason for having user stories is to facilitate discussions between the developers and the business about what it is they think they need. The requirement itself is captured on a 3x5 card that can be discarded on a moment’s notice if a "story" (requirement) changes.  At the end of a Sprint, the business can see functioning software and can tell immediately if what was delivered by the team, was the thing they intended to be built or something different. Feedback is shared and corrections can be applied in the next Sprint. 

Agile does all the same things as waterfall: requirements gathering, design and release planning, construction and testing. But it mitigates risk in several ways.  First, it shortens the cycle for feedback to the length of your Sprint (usually 2-3 weeks, instead of months) Second, it involves the business intimately with the design, development, and construction through each process rather than at the end of each process making it easier to accommodate change. Third, it delivers function software at the end of each Sprint.  So the business has something that works.  If you halt a waterfall project before the construction phase is completed, you have piles of documentation that can't DO anything. 

Agile is the way to go because it minimizes risk to the business by emphasizing communication and constant feedback while delivering functional software.  This is something the business cannot duplicate with weekly Status Reports.

Agile software development eliminates the unnecessary artifacts that are often required to give some architect or business person a warm fuzzy and replaces it with intimate involvement in the project.  It moves quickly and embraces change. 

In a business world that has become almost wholly dependent on technology and technology that changes at incredible speeds, Agile is simply the safest thing for a business to do.

"The environment that nurtures creative programmers kills management and marketing types - and vice versa."

I saw this while reading another blog: Fuzz Box.  So good I had to repost it:

---

"The environment that nurtures creative programmers kills management and marketing types - and vice versa. Programming is the Great Game. It consumes you, body and soul. When you're caught up in it, nothing else matters. When you emerge into daylight, you might well discover that you're a hundred pounds overweight, your underwear is older than the average first grader, and judging from the number of pizza boxes lying around, it must be spring already. But you don't care, because your program runs, and the code is fast and clever and tight. You won. You're aware that some people think you're a nerd. So what? They're not players. They've never jousted with Windows or gone hand to hand with DOS. To them C++ is a decent grade, almost a B - not a language. They barely exist. Like soldiers or artists, you don't care about the opinions of civilians. You're building something intricate and fine. They'll never understand it.

BEEKEEPING

Here's the secret that every successful software company is based on: You can domesticate programmers the way beekeepers tame bees. You can't exactly communicate with them, but you can get them to swarm in one place and when they're not looking, you can carry off the honey. You keep these bees from stinging by paying them money. More money than they know what to do with. But that's less than you might think. You see, all these programmers keep hearing their parents' voices in their heads saying "When are you going to join the real world?" All you have to pay them is enough money that they can answer (also in their heads) "Geez, Dad, I'm making more than you." On average, this is cheap. And you get them to stay in the hive by giving them other coders to swarm with. The only person whose praise matters is another programmer. Less-talented programmers will idolize them; evenly matched ones will challenge and goad one another; and if you want to get a good swarm, you make sure that you have at least one certified genius coder that they can all look up to, even if he glances at other people's code only long enough to sneer at it. He's a Player, thinks the junior programmer. He looked at my code. That is enough. If a software company provides such a hive, the coders will give up sleep, love, health, and clean laundry, while the company keeps the bulk of the money.

OUT OF CONTROL

Here's the problem that ends up killing company after company. All successful software companies had, as their dominant personality, a leader who nurtured programmers. But no company can keep such a leader forever. Either he cashes out, or he brings in management types who end up driving him out, or he changes and becomes a management type himself. One way or another, marketers get control. But...control of what? Instead of finding assembly lines of productive workers, they quickly discover that their product is produced by utterly unpredictable, uncooperative, disobedient, and worst of all, unattractive people who resist all attempts at management. Put them on a time clock, dress them in suits, and they become sullen and start sabotaging the product. Worst of all, you can sense that they are making fun of you with every word they say.

SMOKED OUT

The shock is greater for the coder, though. He suddenly finds that alien creatures control his life. Meetings, Schedules, Reports. And now someone demands that he PLAN all his programming and then stick to the plan, never improving, never tweaking, and never, never touching some other team's code. The lousy young programmer who once worshiped him is now his tyrannical boss, a position he got because he played golf with some sphincter in a suit. The hive has been ruined. The best coders leave. And the marketers, comfortable now because they're surrounded by power neckties and they have things under control, are baffled that each new iteration of their software loses market share as the code bloats and the bugs proliferate. Got to get some better packaging. Yeah, that's it."

- by Orson Scott Card

Building Software the Chicago Way

In 2000-2002 I worked for a company in the Chicago area with a call center in Phoenix. They sent me out to Phoenix twice for training and to work with our Call Center Reps on the software I was developing.  Each time I stayed over the weekend to golf so I was around from Friday until Sunday both times.  I bring this up because in Chicago we have two seasons: winter and construction.  But construction also takes place in Phoenix.  It's just unobtrusive and efficient. At least it was back then. 

The difference between Agile and Waterfall Development is very similar to the difference in approaches to road construction in Phoenix vs. Chicago.

The Chicago Way

In Chicago, huge plans are drawn up years in advance. Billions in capital are approved and funded. The construction project then starts by blocking off the lanes and tearing up everything for the length of the projects.  Traffic patterns are changed for months.  Temp lanes opened, lanes adjacent to construction shut down and people traveling in both directions are subject to the torture that is Construction Season.  For 6,9, 12 months or more, the entire project under construction is ripped up.  At least the Phase 1 of construction.  Usually multiple phases are involved. Each phase taking a year or so. 

All planned up front.

The end result is that by the time the project is actually finished (usually 4-5 years later) the population density has changed as well the traffic patterns and the project that was planned no longer meets the current needs. So they start the whole process over again.  They never catch up to current needs.  And we live under perpetual construction.  First get the guys to tear up the road.  All of it. Then a few months later bring in the guys to lay dirt and gravel.  Then the guys come to flatten it.  Next they lay down the asphalt.  Finally at the end of the project the guys come by and lay down the paint on top of it all.  The funding needed for the project is in the billions.  Cost overruns abound. And whatever is delivered is already obsolete.  Nice.

Contrast that with...

What I saw in Phoenix:

On Friday night after rush hour a crew comes out and blocks off the highway between two entrances and exits.  Nothing else is touched.  Traffic over the weekend is rerouted off the highway and allowed back on at the next entrance.  A crew tears up a piece of road from between exits on Friday night.  Saturday during the day they lay down the foundation of the dirt and gravel and flatten it.  Saturday night they lay the asphalt and flatten it again.  Sunday during the day they add more and flatten it again and let it set up.  Sunday evening they apply paint, clean up their gear, and let the paint dry.  They are gone before Monday morning rush hour.  Exits opened.  New road laid down.  Finished.  The Following Friday they move down to the next exit/entrance and repeat the process.  Instead of billions of dollars being tied up into the project they have hundreds of thousands.  If anything comes up during the reconstruction of the road it can be halted at any time without impacting travel. 

See the difference? At the end of their weekend sprint, the Phoenix team has delivered a completely functional piece of road.  Sound familiar?  If priorities change, they can simply stop moving forward without any lingering issues or unfinished business with which to deal.  Risks are greatly mitigated.  Now imagine if a highway project in Chicago were cancelled in the middle, the workers just getting up and walking away?  What kind of mess would be left behind and headaches?  None of that is a problem if Phoenix decides to halt the project.  The risk in their approach is incredibly small compared to the Chicago Way.  Fewer people are inconvenienced and only for a very short time – and never during the work week when those road are most critical.  There is less road rage, less stress.  And at the end of every weekend, a functioning piece of newly repaved road is available to the end users.  See where I’m going?

Shops that are using waterfall are trying to build applications the Chicago Way.  The only one who wins are developers and contractors. And the politicians they pay for.  That is the Chicago Way.  Commuters, motorists, taxpayers, businesses, deliverymen,  basically everyone else not connected, loses.  And even when the project ends they still don’t win because the requirements have changed.  We need to scrap that approach. We need to bring in Sean Connery and change the way we build software or we will fail much more often than we win.