A Conservative government must be Agile

Everyone knows there is a problem with Government IT projects. They always overrun, on both budget and time. Opposition politicians beat the Government with a stick on this matter, but rarely do we hear what the solution should be. Nor do we hear serious analysis of why such failures occur.

The reason, and the solution, I believe, lies with development methodology. The vast majority of large Government IT projects take the classical "waterfall model" to software development. This is based on the idea of software development progressing like a flowing waterfall through its stages.

You start with requirement analysis, then design, then implementation, then testing, then integration and finally into maintenance (basically, operations, or what some might call Business As Usual). This process is at the core of virtually all IT projects on the scale that the Government IT attempt.

The beginning of the project is all about paperwork and lengthy amount of times without actually delivering anything. By the time you actually reach implementation it's likely that the Government has changed, budgetary priorities have changed, or even, in the worst cases, requirements have changed.

In essence the waterfall method represents the classic grand theoretical narrative models that are inherent in the Left's view of the world. It bases development on the weak assumption of "all things being equal".

The result is that you have projects that underestimate cost, underestimate time, and generally fail to deliver. They create an albatross around the neck of the Government of the day, or, potentially, an incoming Government.

The next Conservative government should not, in my view, allow itself to fall into the trap of grand theory methods of development. Obviously there will be hangover projects that it will not be able to do anything about, but any new IT projects should apply traditional conservative approaches to change.

The implications for Government IT projects under a Conservative government are therefore simple. There should be a strong, and advocating debate in the promotion of Agile development methodology. Agile is, as it suggest, a flexible approach to software development which has the mitigation of risk at its very core.

Software is developed along the approach of rapid iterations of code that deliver little and often within short timeboxes. Each iteration, which last between 1 and 4 weeks, is taken on the basis that it is a small IT project of it's own, which delivers only known and measured necessary functionality to a core code base.

During each iteration, you plan; draw up requirements from user case stories; design; code and then finally test. Every iteration, whilst not necessarily containing a full release, should be able to be released anyway. That is to say the code compiles, and works within the scheme of the iteration's planning, design and requirements.

This creates a lightweight approach to software development which has a greater tendency to deliver on time, and usually in budget. By removing the top-down rigidity of the waterfall method across the entire project life cycle, Agile provides developers with the means to create code fast and also to be flexible if goalposts shift.

If the budget changes, or if new requirements appear, they can be factored in quickly within the next iteration. This approach, whilst appearing initially to be one founded on progression, is, fundamentally, conservative. It rejects the narrative approach by acknowledging the complexity of the world external to the software development. As things can change outside and are thus unknown, the approach mitigates risk in order to accommodate those external unknowns.

It is possible to have a successful Government IT project. A Conservative government just needs to reject the top-down heavyweight approach of the waterfall, and start applying the conservative principles inherent in Agile, which provide for flexibility and, most importantly, delivery.

Anyone interested in a reading a case study of an Agile project (which I happen to still be working on) should look here.