Saturday, November 15, 2008

The Exception-Tolerant Organization - Part 3

As explained in my introductory post on the Exception-Tolerant Organization, ETOs emphasize their strengths and compensate for weaknesses by working together and communicating openly. If the concept sounds simple, that's because it is simple. But with all of the advanced ways, means, and tools we have at our disposal to collaborate and communicate openly, we still find in today's world places and situations where these fundamentals are simply not done.

Working together involves several cultural practices within an organization:

- Total team involvement from the start
- Proactively seeking the improvement of the organization
- Staying open to new ideas and possibilities
- Shared goals without harmful personal agendas

The first point is easier done than said, but the other three involve a resistance factor to change within our organizations. ETOs have the cultural grounding to foster and promote all four points above. As discussed in Part 2, ETOs systematically implement the communication pathways and processes to support these points.

But as is often the case in life, the greater challenge in overcoming obstacles can come from within ourselves, especially on the last point above. We don't always realize that we are more in competition with ourselves than we are at odds with others, and many times we are better served improving ourselves and overcoming our own shortcomings. Instead, we often put up walls where requirements and miracle solutions are volleyed back and forth, neither ever really satisfying the concerns or goals of the parties on both sides of the wall. But when we take down the walls and mprove ourselves, our best foot is then truly placed forward for the benefit of our organizations. Our strengths become the organization's strongest capabilities to be deployed across the greatest spectrum of benefit. And our weaknesses are compensated for by our continuous self-improvement, by identifying risks early, and especially by communicating openly.

Communicating openly, at its most basic level, involves face-to-face discussion, debate, and sometimes conflict - something that people can tend to avoid, in both their personal and professional lives. But for open communication to be effective, an environment needs to be created by the organization where intense debate and disagreement are tolerated, and conflicts can be satisfying to resolve. The organization should make it clear that it is okay to disagree and debate issues, but the result of each debate should still include a clear decision to move forward along a certain path of action.

As an exercise, have everyone in your organization begin to answer these questions out loud daily. Ideally, everyone related to a project or a core business of your organization should be in the same room (or on conference if necessary):

- What did I accomplish yesterday?
- What am I working on today?
- What obstacles or problems am I facing?
- How confident am I that the current goals will be accomplished on-time?

For the last question, use some kind of rating scale. Example: have each person rate their confidence from 1 to 5, with 5 being the most confident. Any answers below a 4 should be a cause for concern and those concerns should be addressed openly. The Agile practice called Scrum advocates the daily use of these questions.

Communicating openly also involves the appropriate use of communication tools. Communications involving urgency or time sensitivity should use direct methods: direct-line phone, internet, and video calls; direct text messages/pages; and of course face-to-face conversation. The use of email and instant messaging, while becoming increasingly mobile and location-independent, should not be relied on for urgent time-sensitive communication. These communication methods often have either multiple inboxes or streams/threads of communication occurring simultaneously, have lengthy queues of messages attached to them, or depend on having your communication device successfully "subscribe" to that message stream. The number of inboxes/threads and the size of the inbox queues are things that you cannot guarantee to be small enough so that your urgent message is received timely. Eliminate this frustration up front by identifying early a reachable line of communication to use for urgent matters.

Some very simple and fantastic exercises in working together and communicating openly (many taking 10 minutes or less to complete with a noisy room full of people) can be found here. While some of these are focused on Agile principles and practices, many of these deal with the core issues of communication and collaboration, and may help to expand your thinking. My thinking was certainly expanded after participating in some of the exercises. My thanks to Michael De La Maza for bringing these to my attention.

Monday, November 10, 2008

Challenging Our Assumptions

At a recent party I sat with a technology team whose members were dejected from having their latest project terminated by their company. This team spoke fondly about their months of development, their early adoption of cloud computing methods and technologies, and even their conferences and consultations with members of NASA regarding the use of advanced technologies. They shook their heads and couldn't understand why the system didn't succeed.

This team had hailed the project as the next great data transmission hub for the team's company. They specified communication protocols and APIs, worked with the application developers to foster understanding, and even reviewed a few sample data streams to understand the problem space. The project seemed to have everything going for it. But during construction of this hub, the team made a fateful design assumption regarding the separation of data streams within communication channels.

This assumption was that data streams could be separated by a data pattern that would "never be seen" in the actual live data itself. The team was warned by the application developers and certain managers that the possibility of this data pattern showing up in live data was in fact likely. Still, the team moved forward with completion of the system, performed some successful testing with a thin time-slice of data transmissions over a period of several days, and then went live with the system.

Within three hours of the system going live, this data pattern delimiter showed up in several places in the live data. As a result, the system was delivering incomplete data and crossing data transmission streams. Processes that were dependent on this new data transmission hub had to be reverted back to their old transmission methods. Beyond changing the data pattern delimiter to something else that might "never be seen" in the actual live data, the team did not have an alternative solution to this problem. And thus a short time later, the project was terminated.

Unfortunately for this team, a fundamental design assumption was made early on that contradicted directly with the problem space that their solution would address. The lack of challenge, investigation, and thorough testing of this assumption sealed the project's fate. While we are often proud of our assumptions, we are made better when our assumptions are challenged and tested at the earliest possible opportunity.

Thursday, November 6, 2008

Project and Technology Development Methodologies

Rather than be the trillionth blogger writing on the subject of Agile, refactoring, Test-Driven-Development, domain-driven-design, and other practices and paradigms, I'll lay out some pragmatic principles to keep in mind when executing projects, no matter which development practices you adopt. In the rush to absorb the hottest trends, we should keep in mind the tried-and-true principles that still work:

Start with iterative and incremental. Project phases are well and good, but in order to complete a project successfully, the path to completion needs to be traversed. And yes, you must have an idea of your destination. But will you wait until all requirements are specified to the last detail before moving on to any architecture or design concerns? Will you have a design project phase that takes into account the current state but fails to connect with the changes in the world three months later when the design phase is completed?

If you break down your project phases into iterations, with a guaranteed feedback session at the end of each iteration, you will be much more likely to keep up with changing requirements and conditions. You will also be able to identify failing efforts sooner, before they cause damage later.

Start as soon as you can. One of the common reasons for project slippage is that some critical component or project dependency was not available earlier in the project cycle. Was it because the need for this component was not forseen? No, it was just not available at the appropriate time. One way to mitigate this is to start working on these components after just enough baseline requirements have been gathered. You may spend a little more up front to support this initial development, but it will cost you far more in the long run should you end up in an ill-timed slippage scenario. The ways it may cost you range from longer development cycles to mis-timed market entry to project personnel turnover. These are big-ticket costs.

Military leaders are trained to make decisions with 40% to 70% of the information necessary, and regularly provide feedback as more information is available. You can do the same on projects and be very effective.

Do your Risk Management. As stated so eloquently in the fantastic treatise on software project management Waltzing With Bears, "Risk Management is Project Management for adults." Listing and valuing project risks is something that can be done to 100% completion up front. But the great (and sometimes painfully realized) thing about Risk Management is that 100% completion is often not enough. Here is a great place to be iterative and incremental in providing feedback loops on the state of risks, mitigating circumstances, and changing requirements. If you build the iterations and feedback loops into your risk management on a project, the rest of your project practice will need to follow suit just to keep up.

You'll also be more inclined to tackle project deliverables that resolve the greatest risks first. The sooner these risks are resolved, the more accurately you can publish a date range for delivery.

Test early and continuously. Your project exists in some form from start to finish. It may start on paper, and live in hardware and software at the finish. But in whatever form it exists, it can be tested. As new requirements are gathered and formulated, test cases can be drafted and pitted against the project's assumptions. As new hardware is installed, its images and monitoring agents can be configured and proven. As new code is written, test cases can be written before or alongside the code.

Don't wait for a project to be 50% or 75% completed to start drafting up test cases and setting up a test facility. Automate much of your tests if you can, so that they can be run at least once a day. Automation is again a case of a little more effort up front, saving much more cost and headache later.