The Plan is not the Objective

This is a write up of a session that I was part of at UKGovCamp11 – I’ve written it as a separate post as I want to take these ideas forward in some way and need to think about how to do this. A few of us have been speaking about the use of Agile in the Public Sector – most notably Public Strategist and Michelle Ide Smith (who live blogged the session here) so we knew we wanted to have two sessions on this – one to focus on the use of Agile for its originally purpose – software development – and the other to explore the aspects of Agile that might be relevant in a reworked Policy process.

Michelle did an excellent job facilitating the first session and if you are interested in this area you should take a look at her slides (I can’t dig out the link to them so will add them next week). We use Agile over at Public-i and as you can read from Ady’s blog this is really a work in progress for us – we are constantly deepening our use and understanding of the process but we have found it invaluable not only as a way of coding more effectively but also as a way of making sure that the development team are focusing on the things that the business needs most acutely. We have loads of improvements still to make but I am very much convinced that this is the right path.

Why Agile?

We tend to treat Policy as a constant – the point of this post is that it shouldn’t be. The objective of the policy is the constant – the policy itself needs to vary in face of changing context and the delivery plan needs to vary even more and we can deliver better outcomes more effectively if we allow this to happen.

I have written before about the need to move from Waterfall to Agile thinking for Policy delivery and Stefan has also written about this here. As a precurser to the wider reading I want to do to support some of this thinking its really worth having a look at this 1985 paper by Fred Brooks where he really introduces the flaws in waterfall thinking: No Silver Bullet — Essence and Accidents of Software Engineering. I also want to catch up with Fred Garnett who mentioned he has already written some related ideas up and had come from a parallel session on agile for learning – so there is definitely something in the air. Next step will be to see if I can get some proper sit down time with Public Strategist to see where we think this could go. If you have any interest or thoughts then please let me know.

In explaining how Agile approaches could help change the way we create and implement policy its worth reflecting on the why the software industry made the shift to agile type methodologies. Its really about three things:

• An increased awareness of complexity and the lack of a definitive answer – – and we need to build for complexity. This means we need incremental progress towards an objective rather than thinking we can capture everything at once
• An awareness of the impossibility of developing anything in isolation – we exist in open not closed systems and no idea exists in isolation – which means we need a process that is not a black box
• A shift from engineering to systems thinking

And of course the fact that everyone hates writing specifications – not the least because you never get them right. There are other more code related motivations that are the results of developers working further and further away from the machine layer but lets not get into that now.

The important thing for me in exploring the connection between agile for code and agile for policy is the fact that policy delivery is failing for much the same reasons. As the purview of the state has increased it is impossible to implement policy without affecting other areas and there is a realisation that we have to start thinking about people’s lives and communities rather than trying to modularise our existence into tidy manageable chunks.

The aim of the session was to try and explore what elements of an Agile software approach could be applied to the policy process and to this end I sketched this to work from:

The sketch seemed to stand up to debate which was good and there was some useful debate. Here’s my take on it:

• Keep the bigger objective in mind – the plan is not the goal: This is vital as far as I am concerned – in that old army adage no plan survives contact with the enemy and any good project manager is highly resilient and adaptive. You need to focus on the objective and adapt as necessary to achieve it. This means that politicians need to set the goal and trust highly agile teams to get on and deliver it. The plan in not the goal.
• New attitudes to risk – break it down and make it manageable: Lots of nodding on this – we consensus was that its better to fail fast and fail early and as Dennis North said – fail cheaply. There are no end of examples where this advice should have been heeded. However we have a political and media culture where failure is not tolerated – which is an impossible position which leads us to either hide or spin all outcomes into something positive and then further fuel the idea that everything succeeds. Tolerating failure is not weakness and we need to find a way to have a more mature debate about this – not only to support wider innovation but more importantly to stop failure on any large project.
• Small incremental improvements rather the big bang: Big switch on implementations are like launching the Titanic and they are a hangover from an engineering mindset which is supported by a media who pander to an attention deficit public (I know that sentence sounds cross – turns out I am cross about this). We can and should be able to see what happens when things are released in manageable stages which we can learn from as we go along. We do not need to crack a bottle of champagne and create a mythical onswitch for a new policy – we need to show constant incremental improvements with no big failures.
• Test as you go along – build evidence rather than theories: Harry Harold quizzed us on the value of testing in the policy process and questioned if the fact that many things just can’t be tested meant that the agile metaphor breaks down here – which was a good point to explore. My view – which seemed to be accepted was that testing is as much about reducing risk as about meeting acceptance criteria. Testing of policy has less effect on risk than testing code as there are fewer things that can be tested but instead I suggest that you take an action research approach and layer and build evidence as the work proceeds. How you do this will depend on the area in which you are working but the idea is to move from the anecdotal and theoretical point at which so many programmes start towards an increasingly robust evidence base.
• Adjust in the face of new evidence: This idea of making evidence gathering part of the ongoing process also supports one of the other principles – that you need to adjust your plan in the face of new evidence. Good ideas can turn out to be the wrong solution and serendipity can happen.
• Everyone is responsible for making it work: As I said at the start – one of the things I appreciate about agile in our business is the fact that it can help communicate strategy to the development team. Agile is essentially a very flat and very teamed based approach which values everyones skills – and as a results increases accountablity.
• Use the users: Agile constantly refers back to the user stories and to the client. We need to keep real people at the heart of the policy process rather than design for a Westminster or even Town Hall bubble.
• Communicate All The Time. Agile relies on a team being kept up to date on progress in all areas every day – because things change everyday. And make sure you communicate failure as a saving.

Some observations

I am not suggesting that we have a scrum master in Downing Street – or that we Kanban road mending in councils (though that might work ?!). I am talking about adapting principles at the moment but process and tools could follow and they probably won’t look like the tools that developers use – as Loulouk pointed out you can’t just knock walls and create whiteboard space in Council buildings – you need to adapt to the environment. You also need to accept that this would by no means the only shift that you need to make – we briefly discussed the need to work more co-productively in order to be able to really ‘use the users’.

There is no point in downplaying how difficult it will be to wean government off big programme thinking – much of the state machine is architected to put large programmes together than then deliver them – its how the funding and structures work at a very detailed level. There is also the need to bundle things up into tidy packages for the press and to link statement A with funding B and result C in a way that does not allow for tactics to change when you learn more. There is a need to allow for the idea that the plan will change and that this isn’t of itself a failure – its a reasonable adjustment to changing context. To allow this we need to allow politicians to be able to be seen to change their minds rather than having a knee jerk accusation of ‘u-turn’ leveled at them.

We probably need to remove artificial uses of time as a constraint – its almost impossible if you are talking social change and instead start to measure progress against our wider objective.

All this looks for a more mature politics than we currently have and a lot more trust and cooperation between politicians than I often observe. However if we are going to work more co-productively because we can’t afford to do anything else then these relationships will of necessity change.

However the fact is that the pressure to change is being created by the social change brought about by the deep embedding of technology in our day to day lives – by the network society. If nothing else the transparency that the social web embodies and that government says it wants to deliver with #opendata means that we will no longer be able to hide our policy programmes in big black boxes that we only open up on launch day. We will be scrutinising not just results but also processes and progress – which means these processes need to be robust and defensible. Armchair auditors will not stop at the results – they will track failure and problems back to the delivery process if they are any good and that puts the way in which we make policy as much in the frame as the outcomes of those policies. This means we need processes which will stand up for a network society.

I will be musing more on this and bothering other people to discuss it. If you came to the session would also be keen to hear whether or not you are happy with this overview. Thank you.