The Plan is not the Objective

This is a write up of a session that I was part of at UKGovCamp11 – I’ve written it as a separate post as I want to take these ideas forward in some way and need to think about how to do this. A few of us have been speaking about the use of Agile in the Public Sector – most notably Public Strategist and Michelle Ide Smith (who live blogged the session here) so we knew we wanted to have two sessions on this – one to focus on the use of Agile for its originally purpose – software development – and the other to explore the aspects of Agile that might be relevant in a reworked Policy process.

Michelle did an excellent job facilitating the first session and if you are interested in this area you should take a look at her slides (I can’t dig out the link to them so will add them next week). We use Agile over at Public-i and as you can read from Ady’s blog this is really a work in progress for us – we are constantly deepening our use and understanding of the process but we have found it invaluable not only as a way of coding more effectively but also as a way of making sure that the development team are focusing on the things that the business needs most acutely. We have loads of improvements still to make but I am very much convinced that this is the right path.

Why Agile?

We tend to treat Policy as a constant – the point of this post is that it shouldn’t be. The objective of the policy is the constant – the policy itself needs to vary in face of changing context and the delivery plan needs to vary even more and we can deliver better outcomes more effectively if we allow this to happen.

I have written before about the need to move from Waterfall to Agile thinking for Policy delivery and Stefan has also written about this here. As a precurser to the wider reading I want to do to support some of this thinking its really worth having a look at this 1985 paper by Fred Brooks where he really introduces the flaws in waterfall thinking: No Silver Bullet — Essence and Accidents of Software Engineering. I also want to catch up with Fred Garnett who mentioned he has already written some related ideas up and had come from a parallel session on agile for learning – so there is definitely something in the air. Next step will be to see if I can get some proper sit down time with Public Strategist to see where we think this could go. If you have any interest or thoughts then please let me know.

In explaining how Agile approaches could help change the way we create and implement policy its worth reflecting on the why the software industry made the shift to agile type methodologies. Its really about three things:

  • An increased awareness of complexity and the lack of a definitive answer – – and we need to build for complexity. This means we need incremental progress towards an objective rather than thinking we can capture everything at once
  • An awareness of the impossibility of developing anything in isolation – we exist in open not closed systems and no idea exists in isolation – which means we need a process that is not a black box
  • A shift from engineering to systems thinking

And of course the fact that everyone hates writing specifications – not the least because you never get them right. There are other more code related motivations that are the results of developers working further and further away from the machine layer but lets not get into that now.

The important thing for me in exploring the connection between agile for code and agile for policy is the fact that policy delivery is failing for much the same reasons. As the purview of the state has increased it is impossible to implement policy without affecting other areas and there is a realisation that we have to start thinking about people’s lives and communities rather than trying to modularise our existence into tidy manageable chunks.

The aim of the session was to try and explore what elements of an Agile software approach could be applied to the policy process and to this end I sketched this to work from:

Agile Policy Sketch

The sketch seemed to stand up to debate which was good and there was some useful debate. Here’s my take on it:

  • Keep the bigger objective in mind – the plan is not the goal: This is vital as far as I am concerned – in that old army adage no plan survives contact with the enemy and any good project manager is highly resilient and adaptive. You need to focus on the objective and adapt as necessary to achieve it. This means that politicians need to set the goal and trust highly agile teams to get on and deliver it. The plan in not the goal.
  • New attitudes to risk – break it down and make it manageable: Lots of nodding on this – we consensus was that its better to fail fast and fail early and as Dennis North said – fail cheaply. There are no end of examples where this advice should have been heeded. However we have a political and media culture where failure is not tolerated – which is an impossible position which leads us to either hide or spin all outcomes into something positive and then further fuel the idea that everything succeeds. Tolerating failure is not weakness and we need to find a way to have a more mature debate about this – not only to support wider innovation but more importantly to stop failure on any large project.
  • Small incremental improvements rather the big bang: Big switch on implementations are like launching the Titanic and they are a hangover from an engineering mindset which is supported by a media who pander to an attention deficit public (I know that sentence sounds cross – turns out I am cross about this). We can and should be able to see what happens when things are released in manageable stages which we can learn from as we go along. We do not need to crack a bottle of champagne and create a mythical onswitch for a new policy – we need to show constant incremental improvements with no big failures.
  • Test as you go along – build evidence rather than theories: Harry Harold quizzed us on the value of testing in the policy process and questioned if the fact that many things just can’t be tested meant that the agile metaphor breaks down here – which was a good point to explore. My view – which seemed to be accepted was that testing is as much about reducing risk as about meeting acceptance criteria. Testing of policy has less effect on risk than testing code as there are fewer things that can be tested but instead I suggest that you take an action research approach and layer and build evidence as the work proceeds. How you do this will depend on the area in which you are working but the idea is to move from the anecdotal and theoretical point at which so many programmes start towards an increasingly robust evidence base.
  • Adjust in the face of new evidence: This idea of making evidence gathering part of the ongoing process also supports one of the other principles – that you need to adjust your plan in the face of new evidence. Good ideas can turn out to be the wrong solution and serendipity can happen.
  • Everyone is responsible for making it work: As I said at the start – one of the things I appreciate about agile in our business is the fact that it can help communicate strategy to the development team. Agile is essentially a very flat and very teamed based approach which values everyones skills – and as a results increases accountablity.
  • Use the users: Agile constantly refers back to the user stories and to the client. We need to keep real people at the heart of the policy process rather than design for a Westminster or even Town Hall bubble.
  • Communicate All The Time. Agile relies on a team being kept up to date on progress in all areas every day – because things change everyday. And make sure you communicate failure as a saving.

Some observations

I am not suggesting that we have a scrum master in Downing Street – or that we Kanban road mending in councils (though that might work ?!). I am talking about adapting principles at the moment but process and tools could follow and they probably won’t look like the tools that developers use – as Loulouk pointed out you can’t just knock walls and create whiteboard space in Council buildings – you need to adapt to the environment. You also need to accept that this would by no means the only shift that you need to make – we briefly discussed the need to work more co-productively in order to be able to really ‘use the users’.

There is no point in downplaying how difficult it will be to wean government off big programme thinking – much of the state machine is architected to put large programmes together than then deliver them – its how the funding and structures work at a very detailed level. There is also the need to bundle things up into tidy packages for the press and to link statement A with funding B and result C in a way that does not allow for tactics to change when you learn more. There is a need to allow for the idea that the plan will change and that this isn’t of itself a failure – its a reasonable adjustment to changing context. To allow this we need to allow politicians to be able to be seen to change their minds rather than having a knee jerk accusation of ‘u-turn’ leveled at them.

We probably need to remove artificial uses of time as a constraint – its almost impossible if you are talking social change and instead start to measure progress against our wider objective.

All this looks for a more mature politics than we currently have and a lot more trust and cooperation between politicians than I often observe. However if we are going to work more co-productively because we can’t afford to do anything else then these relationships will of necessity change.

However the fact is that the pressure to change is being created by the social change brought about by the deep embedding of technology in our day to day lives – by the network society. If nothing else the transparency that the social web embodies and that government says it wants to deliver with #opendata means that we will no longer be able to hide our policy programmes in big black boxes that we only open up on launch day. We will be scrutinising not just results but also processes and progress – which means these processes need to be robust and defensible. Armchair auditors will not stop at the results – they will track failure and problems back to the delivery process if they are any good and that puts the way in which we make policy as much in the frame as the outcomes of those policies. This means we need processes which will stand up for a network society.

I will be musing more on this and bothering other people to discuss it. If you came to the session would also be keen to hear whether or not you are happy with this overview. Thank you.

  1. Pingback: UKGC11 – thank you « Curiouscatherine’s Blog |

  2. Tom Chance

    January 24, 2011 at 9:43 am


    Some more good thoughts on this theme.

    What I’m struggling with – as a researcher for London-region politicians – is how this differs from current practice.

    To pick one example, the London Plan team have a motto “plan, manage, monitor” to which might be appended “and revise”, describing their very iterative process based on evidence and open discussion, and trying hard to make everyone involved in a common set of objectives across policy silos. Sure, it’s a pretty slow process and the set of “involved users” doesn’t include 99% of Londoners (only planners, relevant experts and lobbying organisations).

    In fact, as I think through my different policy specialisms I can think of examples all the way from the relatively monolithic London Plan through to specific objectives being pursued in a fairly agile, incremental way, and I’m not sure how I would describe your approach to a seasoned wonk in a way that would make it sound fresh.

    As somebody who is half geek, half wonk, I’m not sure where spec/agile difference maps onto the policy world. Can you help with examples?


    “All this looks for a more mature politics than we currently have and a lot more trust and cooperation between politicians than I often observe. However if we are going to work more co-productively because we can’t afford to do anything else then these relationships will of necessity change.”

    This reminds me of the idealism of orthodox C19th communists… the crisis will of necessity come and the proletariat will take power! Or indeed of the frustrations of environmentalists who struggle with the inability of politicians to react adequately to sound science even on subjects of the utmost importance such as the continued survival of civilisation as we know it.

    Only political necessity drives that kind of political change!

  3. fred6368

    January 24, 2011 at 3:55 pm

    Hi Catherine,
    here are some follow ups, both linking to our Architecture of Participation blog;
    On Agile Policy: A summative post of where we are on this now Public Value 2011;
    We have been working with some communities to develop the Policy Forest, a tool for identifying Policy outcomes at events. Our best write-up is on the JISC Sustaining Innovation event last summer;
    I should have put this up for #ukgc11 at the Open session. Will do this next time.
    Given that Tom says you are saying nothing new compared to the super policy development process we now have in place I will add two points.
    1) I see Agile as making the policy development process more participatory
    2) The Policy Forest technique is about capturing expertise from a professional Community of Practice & feeding that into a policy process
    I mentioned our Policy 2.0, which builds on AoP and Policy Consultation work we did for the DfES and my co-writer Nigel Ecclesfield is digging it out, in which case we will post it up on the AoP blog

  4. Christine Crandell

    January 24, 2011 at 5:44 pm

    Hi John,

    Great post. I’d like to expand on the point of “communicate all the time.” As organizations writing policy to serve the needs of the public, government agencies have even more of a need than anyone not just to communicate with their own team, but to constantly communicate with their constituents.

    Most political communications are somewhat of a marketing/campaigning effort. To make substantive improvements in Agile policy developments, government needs the tools and processes in place to systematically incorporate constituent feedback into policy development. Touring the city on a photo-opp and speaking to residents one-at-a-time, coming back to the office and saying “our constituents want this” just isn’t cutting it compared to what’s possible.

    Online collaboration and social communities make it possible for government to implement the tools and processes to systematically identify what constituents want and allow those constituents, experts, lobbyists and politicians to share their views, expertise and wants in an online collaboration. This could really spark a lot more cooperation in the political process and arm politicians with more meaningful information directly from all the involved parties to develop better policy, faster.

    -Christine Crandell, Accept Corporation

  5. Carl Haggerty

    January 25, 2011 at 2:20 pm

    Great post Catherine,

    I’m fascinated with the potential but sitting and working in the culture I think I need to start thinking about what the first steps actually are and how I might evaluate my authority to ensure that the first step triggers the most change in this direction.

    Looking from my current viewpoint – i see my organisation on the brink of unprecedented change towards a more flexible and agile model (theory). I’d personally like to see these kinds of methods embedded into the new emerging culture to ensure some sustainability. This just feels like the right time to start sowing the seeds and tackling this.


    • curiouscatherine

      February 6, 2011 at 9:22 pm

      Hi Carl – have been thinking about what you are saying in terms of instigating and supporting innovation – and that important first step

      Am increasingly drawn to the idea of a more focused unconference style event – with the idea of outcomes and actions embedded in the format.

      We’re doing something along those lines I hope with CityCamp Brighton – will let you know how it goes!!!!


  6. Pingback: Public Value 2011 « Architecture of Participation |

  7. Pingback: Crime mapping and why I will never again go for a big bang launch « Public-i Blog |

  8. Pingback: Agile vs PRINCE in Local Government |

  9. Pingback: Agile for communications « Sharon O'Dea |

Leave a Reply

Your email address will not be published. Required fields are marked *