Agile Advisor

So Long and Thanks for the All the Manifestos

David Starr traces the evolution of the agile movement from inception to today's "post-agile" practices.

• By David Starr
• 06/16/2013

I've long held that the original Agile Manifesto (actually called the Manifesto for Agile Software Development), created in 2001, was an apology. It was a recognition that so many in our field were failing to serve our customers and organizations, instead treating them with indifference. The Manifesto signatories took the rather logical leap of observing the outcomes of their work, rather than focusing merely on parts of the pieces that made the whole, as we engineers are prone to do.

To our industry's good fortune, it caught on. Big time.

The ensuing rush of passionate debate, newfound passion for the craft and dissatisfaction with the status quo led to two primary agile discussions:

• Technical Stuff: What engineering practices are most valuable?
• Human Stuff: How might teams of people making software best work together?

Technical Stuff
"We're tired of making crap," was an often-heard refrain during initial agile debates. Accordingly, most agile engineering discussions focused on keeping quality high throughout a complete development cycle. Agile asserts that, rather than allowing buggy code to "stabilize," you should ensure software doesn't need such a period of time before you can trust it. You must keep the software healthy at all times.

In pursuit of this, the engineering practices of software development entered a new era. Test-first practices evolved to being a de facto design tool. Continuous integration (CI) grew from a novelty to a basic measure of professionalism. Pair programming found occasional root. Formal modeling fell into disfavor for all the right reasons. The understanding and use of design patterns resurged. This coalescence of engineering practices was long overdue, and took root in the agile movement, which itself sparked from the Agile Manifesto.

While much of the industry was embracing agility with arms wide open, others saw agile practices as risky, amateurish and something to be endured by those not "serious" about engineering. This polarization undoubtedly slowed the pace of legitimately valuable technical practices. Many beneficial practices associated with agility are valid and useful in any waterfall project. Unit tests work just fine in a plan-driven project, for example.

Human Stuff
While the technical-practices discussion was thriving, so was discussion about the more human issues. Ideas such as user stories, backlogs, Scrum, extreme programming (XP), acceptance-test-driven development (ATDD), retrospectives, just-enough process and self-organizing teams rushed onto the stage. The noise and fury generated on this side of the agile discussion undoubtedly slowed adoption of the technical practices that might otherwise have been accepted as merely logical.

On this side of the discussion, certifications and various branded agile methods arose. Looking back, this was undoubtedly a necessary evolution in the industry, as developers tried to bridge agility to more mature ecosystems. In the interest of full disclosure, I made my living on this side of the discussion for several years. I wore the white robe of theoretical certainty, although it didn't fit terribly well.

The religions of "iterative and incremental" and "continuous flow of value" continue to battle for attention, but most techniques are relegated to being arrows in the same quiver of enterprise project management. Kanban, Scrum and even plan-driven models sit alongside each other in most companies, waiting to be applied to contextually appropriate work. This seems to be working just fine.

Good-Intentioned Drama
With the inarguable success and reach of the Agile Manifesto, other grand proclamations soon followed. By the mid-2000s, agile was on a roll.

The Declaration of Interdependence: This manifesto-like document focused on project-management leadership. Many of the authors were also Agile Manifesto signatories. This attempt to provide a beacon of hope to project management professionals never really caught on. The eventual ambiguity of the term "agile" meant that the concerns of these professionals were accommodated in the original discussion.

ALT .NET: David Laribee coined the term "ALT .NET" as a handle for the community of passionate Microsoft developers who were sick of the old ways of doing work, both from a technology and human standpoint. There was goodness in the ALT .NET space for a while, and the discussion drew respected thinkers like Jeremy Miller, Roy Osherove and Scott Bellware. (There's even still a podcast archive out there worth a listen.)

Eventually, the conversation turned angry. The community-created "ADO.NET Entity Framework Vote of No Confidence" typified this anger. The tone this struck eventually drove Scott Hanselman to famously posit at the 2009 Seattle ALT .NET gathering, "Why so mean?" This seems to have popped the balloon.

This occupy movement faded, just as the one on Wall Street has done in recent months.

The Manifesto for Software Craftsmanship: Yet another frustrated community of software developers raised their collective hand circa 2009, identifying with the term "Software Craftsmanship." The resulting Manifesto for Software Craftsmanship reads like a call to action for artisans and tradespeople.

This community is rehashing some of the same ground covered by ALT .NET, but includes developers from all stacks. The discussion is still active and evolving, but isn't changing as fast as the technology itself, leading me to wonder if this conversation has lasting legs.

Post-Agile
Thankfully the meanness hasn't stuck, while much of the agile goodness did. There's just no reasonable argument to make against better communication, higher quality and responsive business practices. The conservative organizations that were late or not present for the original agile discussion are now interested in the ideas that have matured in the agile community. The business climate has genuinely changed, and even today's bureaucracies understand they must respond quickly and deliberately with added value in their products and services.

Software developers needed the lessons of the methodology wars to find their focus. Conversations today focus on the effectiveness of teams and organizations making software. To that end, I teach to the ideal, coach to the pragmatic and perform to the possible.

I believe this new face of agility has three simple tenets, and doesn't need a manifesto at all:

1. Ship as often as possible.
2. Keep quality high.
3. Solicit and respond to feedback.

Ship Often
I've yet to find a bad software situation that isn't improved by delivering working software more often. Doing so requires a healthy mixture of engineering practices and work-management techniques. While many canned recipes exist, most teams season to taste without disaster. Shipping often offers amazing business opportunities, but requires discipline and a sense of what is appropriate for a given situation.

A lean startup is likely more able to ship frequently than the IT department of a Fortune 100 bank. Issues such as scale of deployment, size of existing code base and even corporate culture influence the frequency of ship cycles. This truth frustrates executives seeking to buy a box of agile fairy dust to sprinkle over their teams. This is the hard work of leadership, and useful shortcuts are rare.

Increasing the frequency with which a team delivers working software has beneficial side effects, even when there's no genuine business case to make for releasing to a customer. Regularly exercising a delivery cycle as completely and as frequently as possible develops a team's skills for actually shipping. This drives the development and maturity of other necessary ingredients, including:

• Frequent code review
• Inspection and adaptation of those doing the work
• Collaboration with stakeholders
• Minimal requirements expressions and inventories

Whether a team uses Scrum, Kanban, waterfall or an agile goulash (the most likely scenario), a lot can be learned from answering two questions:

1. How long would it take to release just one comment in one code file to customers?
2. How can that time be cut in half?

Keep Quality High
Software can't genuinely ship frequently if its quality is low. Too much energy is spent bending existing code to one's will instead of adding new value. As lines of code increase, so does the time required to wrangle those lines into production, and the goal is to make that growth curve as flat as possible.

The best teams I've seen keep quality high by imposing many different practices on themselves. Not all of the following practices are performed by all teams, but those that regularly ship with high quality have most of these basics in place:

• CI
• Automated deployments
• Test-first practices in all their variations
• Test automation
• Respect for code-analysis rules
• Code metrics with agreed-upon target measures
• Peer code reviews

Each of these practices is a study unto itself, but most of you generally know of their existence. The key to succeeding with them is to stop reading about these things and try them. Then improve. Without them, organizations won't speed time-to-market for new software.

Solicit and Respond to Feedback
Once a team is able to ship frequently with high quality, it's time to add the true magic ingredient of business responsiveness. Asking customers and users for their opinions and feedback isn't enough. The software must actually change by taking that feedback into account.

Too many marketers, product owners, entrepreneurs and even development teams give lip service to feedback, but listen only to that which supports decisions they've already made. It's easy to forget the real value is in acting on feedback, not just being open to hearing it.

Arguably the most insightful way to solicit feedback is to actually measure user behavior. More and more software is building experimentation capability in from the start. For example, a Web site may offer users two ways to accomplishing the same task, and measure the usage or success rates of each. With this new understanding in hand, it becomes clearer in which feature to invest and which to go fallow.

Regardless of how it's done, listening and then acting on learning is key to keeping up. At least monthly, software teams need the ability to:

1. Show their work.
2. Ask, "What do you think?"
3. Make a list of things to do in response.

Simple, But Not Easy
The agile discussion has moved beyond the need for the manifesto approach. It's clear that agile practices are better for delivering most software projects than plan-driven techniques. While the relative merits of agile methodologies have become an exercise in navel gazing by those who rarely make software anymore, the real-world developers are getting back to work.

Whether a team uses Scrum, Kanban or rolls its own techniques, more are embracing the new agile. This "post-agile" world has less dogma and more pragmatism. Post-agile is summed up as:

1. Ship often.
2. Keep quality high.
3. Solicit and respond to feedback.

It sounds simple, and it is. It sounds easy, but it isn't.