Tag Archives: scenario planning

Planning should not require a Gantt chart

There’s a standard slide in my bag of tricks which finds it’s way into a surprising number of presentations. It’s a simple slide, one allowing me to explore the idea of planning as a spectrum of possible methodologies rather than treating planning as an either-or choice: either be bottom-up reactive, or requiring us to engage is a major top-down planning processes. By combining elements of both top-down and bottom-up approaches we can modulate how reactive, or how proactive and predetermined, our execution is. The idea is to set the slider where we want it, at the position that best suits what we are trying to achieve, instead of being forced to one end of the spectrum or the other in a purely reactive or purely deliberative mode.

A slide for our times
A slide for our times

Planning has a bad name. How many projects get stuck in that endless spiral of planning (and re-planning) as they try to find their way to the middle of the project hedge maze? It doesn’t seem to matter if we’re planning the expansion of our business into a new geography or a major IT transformation, the process drives us all nuts; wasted days and nights spent constructing (and then deconstructing and reconstructing) elaborate Gantt charts which we’re all sure will never be used.

All of this runs counter to what could be called the first rule of management:

Make a timely decision, and then make it work.

Most organisations’ first thought is to focus on being reactive and deal with the problems of here and now. Unfortunately this too has its own challenges. The Agile software movement can be seen as one well documented reactive methodology which has seen mixed success. Where time, scope and cost are negotiable, taking a reactive approach is the same as choosing to put scope up for grabs. The result of the project will be whatever the project manages to deliver, unsurprisingly. This might be enough to realise the original business case, or it might not.

A top-down, deliberative, approach is planning as a chess game (the untimed[1]Types of chess games defined at Chess.net rather than the timed variety). We assume that we have complete information and can take as long as we want to formulate our next few moves. The more we know about our opponent — the more tacit knowledge we have about what might go wrong (and right) on a project — the better and more accurate our plans will be and the further into the future we can plan. We can measure and evaluate our current situation, use tools like scenario planning[2]Martin Börjesson has put together some nice resources on Scenario planning to anticipate the future, or even hire an innovation consultancy to identify our next, disruptive move. In a perfect world the only limitation to our planning ability is our own ability to comprehend the problem at hand (or the number of tasks we can cram into a Gantt chart, whichever comes first).

The problem is the unexpected — the unpredicted (and possibly unpredictable) event — which makes our plans unravel. It’s the event that we didn’t manage to anticipate, invalidating our assumptions and converting a carefully crafted plan into wasted effort. The faster the environment changes the more likely it is that something unexpected will happen between the plan’s conception and the delivery of its final outcome. Even worse, we might not even finish planning before something unexpected pops up which invalidates our work to date, causing us to restart the planning process.

Take your average major IT transformation: a multiyear project involving a massive investment in both time and money. To kick-start the project we need to predict what the business will need when the transformation is finally delivered (and as I’ve said elsewhere, this is a challenge in itself[3]I’ve already told you 125% of what I know @ PEG). If we’re lucky, then assumptions we’ve made about the business’s requirements will still hold true all the way through to the end. However, what’s more likely to happen is that the change orders will start arriving well before we’ve finished, possibly even we’ve completed requirements gathering. The team then struggles to divide their time between scheduled work and change orders before re-planning (and often re-planning again and again) to try and regain control of the project. Try as we may though, the end result is often more like coughing up a fur ball than the slick solution we originally imagined.

A reactive approach, on the other hand, doesn’t worry having about complete information. The focus is on reacting to events as they are encountered, with little thought to long term goals or strategy. A shotgun approach, as it were. We craft our strategy by creating a set of rules covering what we should do in all the circumstances we care about. See a stop sign? Stop. And so on. The challenge is to craft a set of rules which cover most eventualities, and this is a major focus of the “agile” methodologies. They try and craft their manifestos and playbooks to document the tactics — the rules — which they think will drive their projects in the right direction.

playbook [ˈpleɪˌbʊk]

  1. (Team Sports / American Football) a book containing a range of possible set plays
  2. a notional range of possible tactics in any sphere of activity

The American Heritage Dictionary of the English Language, Fourth Edition.

The unexpected is not a problem for a reactive approach: you simply deal with problems as they arrive. If you’re not expecting anything in particular, then whatever turns up must have been what your were expecting. (There is logic in there.) However, this puts you at the mercy of what does turn up; as you’re focused on reacting to events, the events which arrive (and the order they arrive in) determine where you end up. While cost and time (i.e. effort) might be fixed, scope (and therefor the final result) is up for grabs, making it hard to drive a reactive approach to planning to deliver a well defined outcome.

With it’s focus on incremental delivery and improvement, a reactive approach is not an obvious bedfellow for your major IT transformation. It’s modus operandi is to focus on shot term demands, understand what the business needs next, build that, and then iterate. The IT estate evolves (somewhat) organically as we add things here, patch things there, and generally muddle our way through. Yes, we are reacting to short term business needs, but at what cost? Its a bit like taking our hands off the tiller to leave ourselves to the mercy of the wind; our boat will probably stay afloat, as long as the weather isn’t too bad, but we have little control over where we end up. Without the ability to drive toward a clear, long term goal, we soon find that all the well meaning tactical decisions we’ve made have resulted in a giant fur ball, just like those occasions where we tried to drive the project forward with a giant Gantt chart.

Neither of these two approaches to planning are necessarily wrong; they’re just not suited to solve the sort of problems we’re dealing with in the new normal[4]The new normal @ McKinsey Quarterly. They represent two extremes — proactive and reactive planning — but we need to balance proactive and reactive, and find a middle path.

First we need to admit that we’re resource constrained. This might be in terms of time, the funds we have available, or even simply the number of people we can usefully bring to bear on a problem. (As Frederick Brooks[5]Frederick Brooks on the Mythical Man Month pointed, it might take one woman nine months to produce a baby, but we can’t expect nine woman to deliver a baby in one month.) And this is in an environment where meaningful business change is measured in months, if not weeks. The effort required to develop our plan needs to fit into the time available, and our approach needs to provide opportunities to regularly revisit our assumptions and allow us to react to changes within and without the project. We need need to be reactive, but not too reactive, as we need to work within an overarching framework so that we can have some confidence that we are working toward our long term goals.

People — that’s you and I — take a more pragmatic approach to planning. If we didn’t, then I couldn’t imagine that we would manage to make it through the day with everything we need to get done and the disruptions that we all deal with. Think about how you went to work this morning. (Or how you went to the local café for a coffee, if it’s the weekend.) You weren’t purely reactive. If you were then I expect you would still be lying in bed trying to sort out you manifesto and playbook, hoping for something to happen which will prod you in the right direction. Nor do you take an exclusively top-down, deliberative approach, popping open the laptop on your knee whilst in bed and firing up your favourite planning tool. You’d probably still be stuck in bed, working at the never-ending ending task of plotting exceptions and eventualities. (What should I do if the case of divine intervention?).

People take a middle road. We establish a clear goal, usually something quite prosaic, like:

Get to work (on time)

We have a clear understanding — a set of beliefs, things we expect to be true — of the context we need to do this in.

  • I’m lying in bed
  • it’s a weekday
  • the alarm has just gone off
  • the weather forecast was for a clear sky and sun
  • I’m feeling a bit pudgy today

We also have a library of plans — short patterns of behaviour — which we know that, either from experience or by design, will drive us toward our goal. For example, to get to work we might usually follow the pattern:

  1. get up
  2. shower
  3. get dressed
  4. eat
  5. commute to the office

where each of the steps in this pattern can also be goals themselves.

We might, for example, have a few different plans for commuting: one each for by car, by bike, via public transport, and so on. When it comes time to commute, we consider our beliefs (it’s sunny and I need the exercise) and choose what we think is the most suitable option (I’ll ride). If the option turns out to be a bad choice (one tyre on my bike is unexpectedly flat) or unsuitable due to some change in the environment (it starts raining when I’m less than a block from home), then you discard the plan and pick another (heading back to the garage to take the car). After all, we hadn’t invested much in developing or executing the plan so we’re not losing a lot by throwing it away.

The twisty path between us and our goal
The twisty path between us and our goal

The gap after each step provides us with time to pause and reflect, consider our goal, and respond to the environment by selecting how best to execute the next step. This ability to revisit our decisions frequently — at each step on the journey between the office and home — allows us to be reactive, but not too reactive, while the knowledge that each step is part of a large plan provides us with the confidence that we’re working toward our longer term goals. The more granular we make our plans the more opportunities we provide to react to changes in our environment, but at the cost of being less deliberative and proactive, and potentially less efficient. The more coarse grained we make our plans, the fewer opportunities we create for being reactive, by it allows us to be more deliberative and efficient in delivery.

By modulating the size of our plans — the size of the effort we commit to — we can set the slider where we want it on that planning scale, with reactive on one end, and top down deliberative. This lets us align our planning with the degree of uncertainty and change we’re seeing in the environment around us.

I’ve been using this approach to plan and manage business and IT strategies for some time, with a great deal of success. Most of the tooling required already exists in the business business world, tools such as strategy maps[6]Strategy map defined @ Value Based Mangement by Kaplan & Norton, mission statements[7]Mission statement defined @ The Centre for Business Planning, and project portfolio management tools. Using these tools we can clearly identify our goals, articulate the tactics we want to use to drive ourselves toward them, and then create a portfolio of work which realises the tactics.

For example, the mission of being a low cost service provider might result in a goal of having efficient operations (which we can measure via a benchmark), and the tactics we can use to deliver more efficient operations might include consolidation (reducing the number of offices, data centres, or event tools). We can realise this need for consolidation by identifying a set of small projects (such as shutting down a small data centre, and consolidating its servers into a major one), each of which delivers one step on the journey. These projects then become part of a larger portfolio of work, with the resources available and business priorities determining which projects are executed next.

Just like the example above of each one of us finding our way to work in the morning, this approach enables us to balance how reactive or proactive we are at any time. We can easily re-proritise in response to business change by updating the mix of projects to be executed, rather than through a major re-planning effort. At the same time, we can be sure that the portfolio of work is driving us toward our long term goals, as each project aligns with an organisational goal. Planning without a major Gantt chart.

Having too much SOA is a bad thing (and what we might do about it)

SOA enablement projects (like a lot of IT projects) have a bad name. An initiative that starts as a good idea to create a bit more flexibility in the IT estate often seems to end up mired in its own complexity. The problem is usually too much flexibility, as flexibility creates complexity, and complexity exponentially increases the effort required to manage and deliver the software. Without any solid guidance on how much flexibility to create (and where to create it) most SOA initiatives simply keep creating flexibility until either the project collapses under its own weight, or the projected development work to create all the services exceeds the available CAPEX budget. A little flexility is good, but too much is bad. How can we scope the flexibility, pointing it where it’s most needed while preventing it from becoming a burden?

The challenge with SOA enablement is in determining how much flexibility to build into the IT estate. Some flexibility is good – especially if it’s focused on where the business needs it the most – but too much flexibility is simply another unnecessary cost. The last decade or so is littered with stories of companies who’s SOA initiatives were either brought to an early close or canned as they had consumed all the cash the business was prepared to invest into a major infrastructure project. Finance and telecoms seem particularly prone of creating these gold-plated SOA initiatives. (How many shelf-ware SDFs – service delivery frameworks – do you know of?)

The problem seems to be a lack of guidance on how much flexibility to build, or where to put it. We sold the business on the idea that a flexible, service-oriented IT estate would be better then the evil monolithic applications of old, but the details of just how flexible the new estate would be were a little fuzzy. Surely these details can be sorted out in service discovery? And governance should keep service discovery on track! We set ourselves up by over-promising and under-delivering.

Mario Batali: Too much is never enough!
Mario Batali

This much was clear: the business wanted agility, and agility requires flexibility. As flexibility comes from having more moving parts (services), we figured that creating more moving parts will create more agility. Service discovery rapidly became a process of identifying every bit of (reusable) functionality that we can pack into a service. More is better, or, as the man with the loud shoes says:

Too much is never enough!
Mario Batali

The problem with this approach is that it confuses flexibility and agility. It’s possible to be very flexible without being agile, and vica versa. Think of a formula one car: they’re fast and they’re agile (which is why driving them tends to be a young mans game), and they’re very stiff. Agility comes from keeping the weight down and being prepared to act quickly. This means keeping things simple, ensuring that we have minimum set of moving parts required. They might have an eye for detail, such as nitrogen in the tyres, but unnecessary moving parts that might reduce reliability or performance are eliminated.

This gold plated approach to SOA creates a lot of unrequired flexibility, this additional flexibility increases complexity, and the complexity becomes the boat anchor that slows you down and stops you from being agile. Turning the car is no longer a simple of tugging on the steering wheel, as we need governance to stop us from pulling the wrong lever in the bank of 500 identical levers in front of us.

It's really that simple!
It's really that simple!

We’ve made everything too complicated. Mario was wrong: too much is too much.

What we need is some guidance – a way of scoping and directing the flexibility we’re going to create. Governance isn’t enough, as governance is focused on stopping bad things from happening. We have a scoping problem. Our challenge is to understand what flexibility will be required in the future, and agreeing on the best way to support it.

To date I’ve been using a very fuzzy “business interest” metric for this, where services are decomposed until the business is no longer interested. The rational is that we put the flexibility only were the business thinks it needs to focus. This approach works fairly well, but it relies too much on the tacit judgement of a few skilled business analysts and architects, making it too opaque and hard to understand for the people not involved in the decision making process. It’s also hard to scale. We need something more deterministic and repeatable.

Which brings me to a friend’s MBA thesis, which he passed to me the other week. It’s an interesting approach to building business cases for IT solutions, one based on real options.

The problem with the usual approaches to building a business case, using tools like net present value (NPV) and discounted cash flow, is that we assume that the world doesn’t change post the decision to build the solution (or not). They don’t factor in the need to change a solution once it’s in the field, or even during development.

The world doesn’t work this way: the solution you approved in yesterday’s business environment will be deployed into a radically different business environment tomorrow. This makes it hard to justify the additional investment required for a more flexible SOA based solution, when compared to a conventional monolithic solution. The business case doesn’t include flexibility as a factor, so more flexible (and therefore complex and expensive) solutions lose to the cheaper, monolithic approach.

Real options address this by pushing you down a scenario planning based approach. You estimate the future events that you want to guard against, and their probabilities, creating a set of possible futures. Each event presents you with options to take action. The action, for example, might be to change, update or replace components in the solution to bring them in line with evolving business realities. The options are – in effect – flex-points that we might design into our solutions SOA. The real options methodology enables us to ascribe costs to these future events and the create a decision tree that captures the benefits of investing in specific flex points, all in a clear and easily understandable chain of reasoning.

The decision tree and options provide us with a way to map out where to place flex points in the SOA solution. They also provide us with strong guidance on how much flexibility to introduce. And this is the part I found really interesting about the approach. It also provides us with a nice framework to govern the evolution of the SOA solution, as changes are (generally) only made when an option is taken: when it’s business case is triggered.

It’s a bit like those formula one cars. A friend of mine used to work for one F1 manufacturer designing and testing camshafts. These camshafts had to fall within a 100,000 lifetime revolution window. An over-designed camshaft was unnecessary weight, while an under-designed one means that you wouldn’t win (or possibly even finish) the race. Work it out: a 100,000 revolutions is a tiny window for an F1 car, given the length of a race.

An approach like real options helps us ensure that we only have the flexibility required in the solution, and that it is exactly where it is required. Not too much, and not too little. Just enough to help us win the race.