© Best Practices in IT Leadership, Mosaic Media, Inc.
Shifting IT from Process to Product
Methodologies for the new millennium
BY BILL SHACKELFORD, EDITOR, Best Practices in IT Leadership

Ever since the chaotic early days of COBOL programming, IT organizations have struggled to get control of the application development process. As systems became more complex, it became obvious that ad hoc methods of building applications weren’t just inefficient—they were downright dangerous.  As the lines of undocumented code multiplied, so did demands for new systems and system enhancements. Maintenance back-logs made it clear that analysts and programmers needed to follow more standardized procedures to make sure that they and their successors could maintain existing systems and build maintainable systems for the future. Structured methods and so-called "water-fall methodologies" were used to clear back-logs, incorporating better requirements gathering, exhaustively detailed specifications for "deliverables," and tightly controlled procedures for managing changes throughout the system’s development life cycle. The point of all this was to create a "repeatable process" for delivering systems.  Admittedly, without these methods, we could not have come as far as we have in building business systems. And we need strictly repeatable processes for building airplanes and space shuttles. Unfortunately in many cases, systems development methods became ends unto themselves, confusing shelves of paper for useful documentation, and ritualistic systems review board actions for honest-to-goodness business decision-making. The more ensconced a methodology, the more removed it seemed both from the business user and from the applications developer. The "user signoff," designed to cast in concrete the system’s requirements, became the deadly embrace which could only be broken by "user acceptance testing"—a long, bitter battle fought by business users to get the systems they really needed. Far too often, methodologies merely exacerbated the user-versus-IT schism, as both sides cynically peppered their files with documentation designed to cover their positions (CYP!), rather than to provide useful support-ive materials. Although this scenario is a bit exaggerated, I’ve seen similar ones—and worse.

Enter the world of the Web
The fiercely competitive world of e-commerce is forcing a re-evaluation of the way we build systems. Who today has the foresight to predict accurately how a system needs to look six months from now? Change is no longer some-thing to be avoided; change is inevitable and desirable. IT can no longer isolate itself from the business user during the gestation period of an application and wait for implementation to reconnect. Instead, the application becomes the joint effort of IT and the business user, and requires ongoing evaluation and adaptation. Data and process models may be useful in the earliest stages of defining the application, but the working, evolving application itself will be the true test of the system’s viability. Does this mean we need to jettison every-thing we learned about methodologies?

Absolutely not! On the contrary, we need to redefine the practices we use and key deliverables we produce as we develop applications. Sometimes called "light methodologies," these practices vary in detail from expert to expert, but have several important similarities. (Kent Beck’s Extreme Programming Explained and Jim Highsmith’s Adaptive Software Development are two widely read resources in this area.) As a result of pressures to develop Web-based applications, many organizations have become interested in light methodologies. But many of these principles apply to conventional application development as well.

Common threads for IT development
Almost every one of the light methodologies includes some variation of the following principles, and suggests ways to put them into practice:

The application is the key deliverable throughout development. As soon as an initial definition is avail-able, the development team concentrates on delivering a working version of the application. Not a prototype, but a working application. Obviously, not all features will be available in "version 0.01," but the business user will be able to see the product as it evolves.

The business user stays involved from project initiation through product roll-out. From the very beginning, the business user is available (and, in the case of  Extreme Programming, is actually on-site with the development team) to clarify business rules, develop test cases, check application builds, and provide ongoing feedback to the development team. This requires the commitment of one or more representatives from key business areas. Ideally, this person will be the one to demonstrate the working versions of the application at JAD or focus group sessions.

Change is inevitable. Change is good. Both IT and the business user need to re-evaluate a product’s features during its development. To do this, the life cycle contains sub-cycles devoted to creating and demonstrating the application in a JAD or focus group session. At each session, the business user may submit changes to be incorporated into the final product. Everyone understands that in the IT/business area partnership, changes will necessitate re-evaluation of costs and schedule. Often, new features will be added and originally planned features abandoned.

Testing is ongoing. Test planning and testing itself occur from the application’s inception through final roll-out. Tests should be automated, and the entire battery of tests run with each new build of the application. This allows for frequent builds, and demonstrates to the entire team that the application is a viable organism at all times. All documentation relevant to the system is carefully scrutinized for clarity and suitability to its purpose, whether it’s system documentation or user manuals and help screens.

Process improvement must be continuous. The team will continually re-evaluate its standards and practices throughout the project, as well as at key milestones in the application’s development. Wherever possible, programmers will simplify and clarify their code, often using principles laid out in Martin Fowler’s ground-breaking text, Refactoring. Teams are dedicated. These methods only work with teams aligned to develop a single product. During a project’s life, the teams should work closely together—at the same location whenever possible. Splitting a team member’s time inevitably erodes the team’s effectiveness. Don’t try to turn one success into dogma. The quest for the completely fool-proof repeatable process has never fully succeeded. Light methodologies invite us to incorporate new ideas and practices continually as we build applications. It would be a great shame to take the newest practices and try to make them the only practices. Taking the best from time-tested methodologies and newer practices, such as light methodologies, can both speed the delivery of your team’s application and make its development a much smoother process.

BILL SHACKELFORD
© Best Practices in IT Leadership, Mosaic Media, Inc.