Tag Archives: KK Pang

Complexity isn’t (at least in enterprise IT)

There’s a lot of talk these days about complexity in enterprise IT. The heterogeneous solutions we’re building today seem more complex than the monolithic solutions of the past. But are they really? I’ll grant you that a lot of what is being built at the moment is complicated. Complex though? I don’t think so. The problem is that we’re building new things in old ways when we need to be building new things in new ways.

I’ve always used a simple rule of thumb when thinking about complexity. Some folk like to get fancy with two and three dimensional models that enable us to ascribe degrees of complexity to problems. While I find these models interesting, my focus has always been on how do I solve the problem in front of me. What is the insight that will make the hard easy? For me, one simple distinction seems to provide the information I need. Is a solution complex? Or is it complicated?

Something is complicated if the model we use to understand the problem requires patches, exceptions to make it work. The model might be simple and well understood, but we’re forced to patch the model for it to succeed when confronted by the real world; we’re adding epicycles. It’s not a complex system, but it is complicated.

Adding epicycles didn't manage to keep the earth at the centre of the universe
Adding epicycles didn't manage to keep the earth at the centre of the universe

On the other hand, something is complex if it’s difficult to develop a consistent model for the problem. While we might have a well understood model, it’s definitely not simple, requiring a great deal of academic and tacit knowledge to understand. There’s no epicycles, but there are a large number of variables involved, and their interactions are often non-linear.

While this binary separation might not be strictly true (the complicated can sometimes be complex), I find that the truly complex problems are rare enough that the rule of thumb is useful most of the time. After all, that’s what a rule of thumb is. The few times that it breaks down, experience comes to the rescue.

Distinguishing between the complex and the complicated is not hard; just look for the epicycles. Planning engines – such as material planning or crew scheduling – are a good example of a complex solutions. Business processes management is a good example of a complicated solution. Psi calculus – the model at the heart of a modern BPM engine – is well understood and BPM engines work as described. However, managing business exceptions is a mess as support for them is tacked on rather than an inherent part of the model. Smashing together psi calculus, transactions and number of other models has resulted in epicycles.

Most of the problems we’re seeing in enterprise IT are complicated, but not complex. Take the current efforts to create IT estates integrating SaaS and public cloud with more traditional enterprise IT tools, such as on-premesis applications and BPO. Conventional approaches to understanding and planning IT estates are creaking at the seems. The model – the enterprise integration patterns – which we’ve used for so long is well understood, but it’s creaking at the seams as we bolt on more epicycles to cope with exceptions as they arise.

Dr. Khee Pang
Dr. Khee Pang

A great piece of advice from a former lecturer of mine always comes to mind in situations like this. As I’ve mentioned before:

If you don’t like a problem, then change it.
KK Pang

Our solution is complicated because we’re trying to solve the wrong problem. We need to change it.

The problem with BPM is that business exceptions are not exceptional, but are simply alternative ways of achieving the same goals. To resolve the epicycles we need to shift the problems centre of gravity, moving the earth from the centre of the universe to a more stable orbit. If business exceptions are not exceptional, then we should simply consider them as different business scenarios, and use a scenario based approach to capturing business processes. The epicycles then melt away.

I think we can use a similar approach to help us with the challenges we’re seeing in today’s IT estates, the same challenges which are trigger some of the discussion on complexity. The current approach to planning and provisioning IT is data centric; most applications are, after all, just large data management engines. This data-centric approach is forcing us to create epicycles as we attempt to integrate SaaS, cloud, and a whole raft of new tools and techniques. The solution is to move from a data-centric approach, to decision-centric approach. But that’s a different blog post.

Innovation and the art of random

A little while ago I was invited to speak at an event, InnoFuture, which, for a mixture of reasons, didn’t end up happening. The theme for the event was Ahead of the trends — the random effect. My take on it was that innovation is not random, it’s just happening faster than you can process, and that ideas are commoditized making synthesis, the creation of new solutions to old problems, what drives innovation. I was pretty happy with the outline I put together for my talk, that I ended up reusing the content and breaking it into three blog posts, rather than letting it go to waste.

Innovation seems to be the topic of the day. Everyone seems to want some, thinking that it’s the secret sauce which will help them (or their company) bubble to the top of the heap. The self help and consulting communities have responded in force, trying to bottle lightening or package the silver bullet (whichever metaphor you prefer).

It was in this environment that I was quite taken by the topic of a recent InnoFuture event when I was asked to speak.

Ahead of trends — the random effect.
When a concept becomes a trend, you are a not the leader. How to tap into valuable ideas for products, services and communication before they are seen as trends, when they are just … random? Albert Einstein said that imagination is more important than knowledge. Let’s open the doors and let the imagination in for it seems that in the current crisis, the right brain is winning and we may be rationalized to death before things get better.

I’ve never seen the random effect, though I have been delightfully surprised when something unexpected pops up. Having been involved in a bunch of companies and projects that, I’m told, where innovative, I’ve always thought innovation was not so much random, as the result of obliquity. What makes it seem random is the simple fact that your are not aware of the intervening steps from interesting problem through to novel solution.

I figured I’d mash together a few ideas that capture this thought, and provide some (hopefully) sage advice based on what I do to deal with random. I ended up selecting:

  • John Boyd on why rapidly changing environments are confusing,
  • Peter Drucker‘s insight that insight (the tacit application of knowledge) is not a transferable good,
  • the struggle for fluency that we all go through as we learn to read,
  • John Boyd (again, but then he had a lot of good ideas) on the need for synthesis,
  • KK Pang (and old lecturer of mine) on the need to view problems from multiple contexts,
  • the need to follow a consistent theme of interest as the only tractable way of finding interesting problems to solve, and
  • my own experiences in leveraging a network of like and dissimilar minds as a way of effectivly out-sourcing analysis.

The result was called Of snow mobiles and childhood readers: why random isn’t, and how to make it work for you. I ended up having far to much content to fill my twenty minute slot, so it’s probably for the better that the event didn’t go ahead, as it would have taken a lot of time to cut it down.

Given that I had a fairly well developed outline, I decided to make it into a series of blog posts (plus my slides these days don’t have a lot of text on them, so if I just dropped the slides online they wouldn’t make any sense). The blog posts ended up breaking down this way:

  1. Innovation should not be the race for the new-new thing.
    Points out that innovation only seems random, unexpected, as you don’t see the intervening steps between a problem and new solution, and that innovation is the result of many small commoditized steps. This ties into one of my earlier posts of dealing with the speed of change.
  2. The role of snowmobiles in innovation.
    Argues that ideas are a common commodity, and that the real challenge with innovation is synthesis rather than ideation.
  3. Childhood readers and the art of random.
    Argues that the key to innovation is to find interesting problems to solve, and suggests that the best approach is to be fluent in a range of domains (sectors, geographies, activities, …) to provide a broader perspective, focus on a line of inquiry to provide some structure, and build a network of people with complimentary interests, providing you with the time, space and opportunity to focus on synthesis.

I expect that these are more productive if taken as a whole, rather than individual posts.

If you look at the path I’ve charted over my career then this is the approach I’ve taken, and my topic of choice is how people communicate and decide as a group, leading me to John Boyd, Cicero, human-computer interaction, agent technology, biology (my thesis was mathematically modelling nerves in a cat), and so on.

I still have the slides, so feel free to contact me it you’re interested in my presenting all or part of this topic.