Running a business is a bit like being the Fat Controller, running his vast train network. We spend our time trying to get the trains to run on time with the all too often distraction of digging the Troublesome Trucks out of trouble.

Improvement often means upgrading the tracks to create smoother, straighter lines. After years of doing this, any improvement to the tracks can only provide a minor, incremental benefit.

What we really need is a new signalling system. We need to better utilise the tracks we already have, and this means making better decisions about which trains to run where, and better coordination between the trains. Our tracks are fine (as long as we keep up the scheduled maintenance), but we do need to better manage transit across and between them.

Swap processes for tracks, and I think that this paints quite a nice visual picture.

Years of processes improvement (via LEAN, Six Sigma and, more recently, BPM) had straightened and smoothed our processes to the point that any additional investment has hit the law of diminishing returns. Rather than continue to try and improve the processes on my own, I’d outsource process maintenance to a collection of SaaS and BPO providers.

The greater scale of these providers allows them to invest in improvements which I don’t have the time or money for. Handing over responsibility also creates the time and space for me to focus on improving the decisions on which process to run where, and when: my signalling system.

This is especially important in a world where it is becoming rare to even own the processes these days.

We forget just how important a good signalling system is. Get it right and you get the German or Japanese train networks. Get it wrong and you rapidly descend into the second or third world, regardless of the quality of your tracks.

I suggest we just haven’t experimented enough with ways to render information in more usable ways, and once we start to do so, it will like take 10 years (the 10,000 hour rule again) before anyone demonstrates real mastery of the techniques involved.

The problem is that our current tooling for information processing is not up to the task at hand. Unfortunately Tim, like most of us, is still trying to find the best way to managed the information load pressing down on us.

Any suggestions?

Posted via email from PEG @ Posterous

After the fact, it’s easy to point to the bits of evidence and claim that someone should have “connected the dots.” But before the fact, when there millions of dots – some important but the vast majority unimportant – uncovering plots is a lot harder.

This is a lot like the challenge we’ve been talking about under the banner of The value of information. How do we make sense of weak, conflicting and volumous signals we see in the environment outside our business, fuse this with strong signals from data inside the business, and create real insight? Granted, sometimes we’re aware of the signals (or at least the shape of their outline) we need to go looking for, much like Tesco’s decision to integrate weather forecasts and historical till information to predict customer demand. In other circumstances, we’re not so sure what we’re looking for. The business equivalent of predicting (and responding to) the underwear bomber might be managing exceptions in a complex, global supply chain, countering a competitor’s new product launch, or supporting a social case worker dealing with a unexpected crisis in a client’s domestic situation.

It’s tempting to create counter measures – prescriptive workflows designed to resolve a problem – to each of these scenarios on a case-by-base basis. Or even just throw up our hands and continue with the tribal processes of old. But, as Bruce points out, this doesn’t work. The challenge with taking action against specific threats is that the terrorist will simply use a new tactic next time, or you’ll be confronted with yet-another situation. Soon you’ll have overloaded your knowledge workers with exception scenarios which only address yesterday’s problems. You’ve started an arms race which you cannot win.

Bruce’s solution, in the context of security, is to integrate information into an operational decision making framework which wards against generic attacks.

What we need is security that’s effective even if we can’t guess the next plot: intelligence, investigation and emergency response.

This prompts me to think of two things:

First, we might need to add third dimension to that figure from Inside vs. Outside: Precision, to compliment Inside/Outside and Information Age. (Here, the engineer in me is going to split hairs over the definitions of focus, precise and accurate.) This new dimension captures how precise our need is. The Tesco example from above prefers precise signals, signal which communicates a single message. The exception manager might require imprecise signal, a derivative communicating a generic message aggregated generated by correlating a number of (in)precise signals. (A note of caution though, is to remember the recent impact of derivatives on the global financial markets.)

Second, we might want to rethink about how we conceptualise and use information information in our business. We currently have a very linear view, with information generation and consumption tightly connected to the stages of our value chain. It would be interesting to see how some of the ideas and frameworks behind the value of information could be fused with a decisioning framework like OODA. This would provide a tool to simplify the (potentially too complex) value of information framework, and realize it in operational work practices.

I’m not sure about the first point, but I expect the second will be fertile ground for further investigation.

Posted via web from PEG @ Posterous

Most interesting business decisions seem to be a synthesis process. We take a handful of data and fuse them to create an insight. The invention of breath strips is a case in point. We can rarely break our problem down to a single (computed) metric, the world just doesn’t work that way.

Most business decisions rest on small number of data points. It’s just one of our cognitive limits: our working memory is only large enough to hold (approximately) four things (concepts and/or data points) in our head at once. This is one reason that I think Andrew McAfee’s cut-down business case works so well; it works with our human limitations rather than against them.

I was watching an interesting talk the other day — Peter Norvig was providing some gentle suggestions on what features should be beneficial in a language to support scientific computing. Somewhere in the middle of the talk he mentioned the Curse of dimensionality, which is something I hadn’t thought of for a while. This is the problem caused by the exponential increase in volume associated with each additional dimension of (mathematical) space.

In terms of the problem we’re considering, this means that if you are looking for n insights to a problem in a field of data (the n best data points to drive our decision), then finding them becomes exponentially harder for each data set (dimension) we add. More isn’t necessarily better. While the addition of new data sets (such as sourcing data from social networks) enables us to create new correlations, we’re also forced to search an exponentially larger area to find them. It’s the law of diminishing returns.

Our inbuilt cognitive limit only complicates this. When we hit our cognitive limit — when n becomes as large as we can usefully use — any additional correlations can become a burden rather than a benefit. In today’s rich and varied information environment, the problem isn’t to consider more data, or to find more correlations, its to find the best three or features in the data which will drive our decision in the right direction.

How do we navigate from the outside in? From the decision we need, to the data that will drive it. This is the problem I hope the Value of Information discussion addresses.

Posted via web from PEG @ Posterous

As one of the CEOs Andy Mulholland spoke to mentioned “I want to know … when I need to focus in.” The CEO’s problem is not more data, but the right data. As Andy rightfully points out in an earlier blog post, we’ve been focused on harvesting the value from our internal, manufactured data, ignoring the latent potential in our unstructured data (let alone the unstructured data we can find outside the enterprise). The challenge is not to find more data, but the right data to drive the CEO’s decision on where to focus.

It’s amazing how little data you need to make an effective decision—if you have the right data. Andrew McAfee wrote a nice blog post a few years ago (The case against the business case is the closest I can find to it), pointing out that the mass of data we pile into a conventional business case just clouds the issues, creating long cause-and-effect chains that make it hard to come to an effective decision. His solution was the one page business case: capability delivered, (rough) business requirements, solution footprint, and (rough) costing. It might be one page, but there is enough information, the right information, to make an effective decision. I’ve used his approach ever since.

Current BI seems to be approaching the horse from the wrong direction, much like Andrew’s business case problem. We focus on sifting through all the information we have, trying to glean any trends and correlations which might be useful. This works as small to moderate scales, but once we reach the huge end of the scale it starts to groan under its own weight. It’s the law of diminishing returns—adding more information to the mix will only have a moderate benefit compared to the effort required to integrate and process it.

A more productive method might be to use a hypothesis-driven approach. Rather than look for anything that might be interesting, why not go spelunking for specific features which we know will be interesting?  The features we’re looking for in the information are (almost always) to support a decision. Why not map out that decision, similar to how we map out the requires for a feedback loop in a control system, and identify the types of features that we need to support the decision we want to make? We can segment our data sets based on the features’ gross characteristics (inside vs. outside, predictive vs. historical …) and then search in the appropriate segments for the features we need. We’ve broken one large problem—find correlations in one massive data set—into a series of much more manageable tasks.

The information arms race, the race to search through more information for that golden ticket, is just a relic of the lack of information we’ve lived with in the past. In today’s land of plenty, more is not necessarily better. Finding the right features is our real challenge.

Posted via email from PEG @ Posterous

Tesco is using external weather data to drive sales

Tesco, the UK’s largest retailer, has started using weather forecasts to help determine what to stock in its stores across the UK.

Traditional approaches to stock management use historical buying data to drive stock decisions. This has worked well to date, but the increasing unpredictability of today’s weather patterns — driven by global warming — has presented business with both an opportunity and a challenge. An unexpected warm (or cold) spell can create unexpected spikes in demand which go unserviced, while existing stock is left on the shelves.

In Tesco’s own words:

In recent years, the unpredictability of the British summer — not to mention the unreliability of British weather forecasters — has caused a massive headache for those in the retail food business deciding exactly which foods to put out on shelves.

The present summer is a perfect example, with the weather changing almost daily and shoppers wanting barbecue and salad foods one day and winter food the next.

Tesco’s solution was to integrate detailed regional weather reports — valuable, external information — with the sales history at each Tesco store. A rise of 10C, for example, led to a 300% uplift in sales of barbecue meat and a 50% increase in sales of lettuce.

Integrating weather and sales data will enable Tesco to both capture these spikes in demand, while avoiding waste.

(Largely adapted from the article in the Times Online.)