Agile Success During Challenging Times

2009 is shaping up to be a challenging year for many organizations. Their people are being asked to do more with less, or do more just to survive in the marketplace. But there is more to being able to survive than just "doing more".

Challenging times require us to expect risks to materialize, be systematically ready to meet exceptions head-on, and advance our adaptability. This is where being agile becomes critically important for those organizations looking to rise ahead in 2009. And yes, even in challenging years, it is possible to succeed.

Here are two Agile success stories from my own experience with challenging times:

Problem: With a rapidly failing data communication pipeline, and a 1/3 reduction in technology headcount, an organization under tight deadlines to restructure its business needed a replacement for the pipeline quickly. With such a sharp reduction in technologists, the replacement pipeline had to exhibit a low-maintenance footprint while making it easy for the remaining technologists to extend its capabilities under the organization's new business structure.

Agile Solution: By including all the technology team members from the start, everyone was clear on what was at stake, and generally what each member's contribution would be. The team analyzed and designed the extensible solution together, proving it first on whiteboard, then taking adaptable technology to construct the first working version in one day - the total time representing a one-week iteration.

Frequent iterations were then used to construct and deploy vital features of the pipeline. From the time that the first version was working, the technologists assigned themselves to the tasks and pipeline features to complete. By meeting daily, and cooperating with the project leader and their project teammates, they "pulled" down the tasks themselves, rather than having tasks "pushed" or forced onto them. This translated into both faster task completion and higher quality of pipeline features.

The extensible architecture allowed the technologists to deploy the replacement pipeline early and incrementally, but swiftly bring the pipeline's features live at the earliest possible moment. The new solution represented a lower server deployment footprint than the older pipeline. And the servers hosting the failing pipeline (targets for consolidation themselves) were able to be decommissioned much sooner than expected, providing an additional much-needed cost savings to the organization.


Problem: An organization has an imminent opportunity to expand its business. But, its daily processing includes too much manual interference to correct data transmission errors. Business principals are often overworked chasing after data problems and manually translating data from one system to the next. Technologists are too often paged overnight to correct data transmission problems, representing a higher cost to the business when they are not available during the business day to correct the next day's problems. The organization's processing capability is unable to keep up with the new business.

Agile Solution: With only a few weeks to spare, all technology and business team members were included at the start, understanding that an intermediate technology solution is needed to sharply reduce the data transmission errors while alleviating the burden on the technologists to make repairs. User stories were created to detail the desired goals of the business, while demonstrating where the technology could assist. The stories were translated into realistically completable tasks, the effort to complete these stories was estimated by the team using "story points", and the stories were prioritized by the team and assigned into weekly iterations. While a release burndown chart driven from the story points was used to display and chart progress to senior management, the user stories were the focal point of the team's understanding of the challenges faced and the scope of the solution.

This new solution, in addition to reducing the amount of data errors and overnight pages, allowed the business principals to be self-sufficient in repairing the remaining data errors. Any new errors identified by the business could be repaired by the system, thanks to the use of user stories as a model for providing a solution to future data errors.

Project and Technology Development Methodologies

Rather than be the trillionth blogger writing on the subject of Agile, refactoring, Test-Driven-Development, domain-driven-design, and other practices and paradigms, I'll lay out some pragmatic principles to keep in mind when executing projects, no matter which development practices you adopt. In the rush to absorb the hottest trends, we should keep in mind the tried-and-true principles that still work:

Start with iterative and incremental. Project phases are well and good, but in order to complete a project successfully, the path to completion needs to be traversed. And yes, you must have an idea of your destination. But will you wait until all requirements are specified to the last detail before moving on to any architecture or design concerns? Will you have a design project phase that takes into account the current state but fails to connect with the changes in the world three months later when the design phase is completed?

If you break down your project phases into iterations, with a guaranteed feedback session at the end of each iteration, you will be much more likely to keep up with changing requirements and conditions. You will also be able to identify failing efforts sooner, before they cause damage later.

Start as soon as you can. One of the common reasons for project slippage is that some critical component or project dependency was not available earlier in the project cycle. Was it because the need for this component was not forseen? No, it was just not available at the appropriate time. One way to mitigate this is to start working on these components after just enough baseline requirements have been gathered. You may spend a little more up front to support this initial development, but it will cost you far more in the long run should you end up in an ill-timed slippage scenario. The ways it may cost you range from longer development cycles to mis-timed market entry to project personnel turnover. These are big-ticket costs.

Military leaders are trained to make decisions with 40% to 70% of the information necessary, and regularly provide feedback as more information is available. You can do the same on projects and be very effective.

Do your Risk Management. As stated so eloquently in the fantastic treatise on software project management Waltzing With Bears, "Risk Management is Project Management for adults." Listing and valuing project risks is something that can be done to 100% completion up front. But the great (and sometimes painfully realized) thing about Risk Management is that 100% completion is often not enough. Here is a great place to be iterative and incremental in providing feedback loops on the state of risks, mitigating circumstances, and changing requirements. If you build the iterations and feedback loops into your risk management on a project, the rest of your project practice will need to follow suit just to keep up.

You'll also be more inclined to tackle project deliverables that resolve the greatest risks first. The sooner these risks are resolved, the more accurately you can publish a date range for delivery.

Test early and continuously. Your project exists in some form from start to finish. It may start on paper, and live in hardware and software at the finish. But in whatever form it exists, it can be tested. As new requirements are gathered and formulated, test cases can be drafted and pitted against the project's assumptions. As new hardware is installed, its images and monitoring agents can be configured and proven. As new code is written, test cases can be written before or alongside the code.

Don't wait for a project to be 50% or 75% completed to start drafting up test cases and setting up a test facility. Automate much of your tests if you can, so that they can be run at least once a day. Automation is again a case of a little more effort up front, saving much more cost and headache later.