Tag Archives: bank

Having too much SOA is a bad thing (and what we might do about it)

SOA enablement projects (like a lot of IT projects) have a bad name. An initiative that starts as a good idea to create a bit more flexibility in the IT estate often seems to end up mired in its own complexity. The problem is usually too much flexibility, as flexibility creates complexity, and complexity exponentially increases the effort required to manage and deliver the software. Without any solid guidance on how much flexibility to create (and where to create it) most SOA initiatives simply keep creating flexibility until either the project collapses under its own weight, or the projected development work to create all the services exceeds the available CAPEX budget. A little flexility is good, but too much is bad. How can we scope the flexibility, pointing it where it’s most needed while preventing it from becoming a burden?

The challenge with SOA enablement is in determining how much flexibility to build into the IT estate. Some flexibility is good – especially if it’s focused on where the business needs it the most – but too much flexibility is simply another unnecessary cost. The last decade or so is littered with stories of companies who’s SOA initiatives were either brought to an early close or canned as they had consumed all the cash the business was prepared to invest into a major infrastructure project. Finance and telecoms seem particularly prone of creating these gold-plated SOA initiatives. (How many shelf-ware SDFs – service delivery frameworks – do you know of?)

The problem seems to be a lack of guidance on how much flexibility to build, or where to put it. We sold the business on the idea that a flexible, service-oriented IT estate would be better then the evil monolithic applications of old, but the details of just how flexible the new estate would be were a little fuzzy. Surely these details can be sorted out in service discovery? And governance should keep service discovery on track! We set ourselves up by over-promising and under-delivering.

Mario Batali: Too much is never enough!
Mario Batali

This much was clear: the business wanted agility, and agility requires flexibility. As flexibility comes from having more moving parts (services), we figured that creating more moving parts will create more agility. Service discovery rapidly became a process of identifying every bit of (reusable) functionality that we can pack into a service. More is better, or, as the man with the loud shoes says:

Too much is never enough!
Mario Batali

The problem with this approach is that it confuses flexibility and agility. It’s possible to be very flexible without being agile, and vica versa. Think of a formula one car: they’re fast and they’re agile (which is why driving them tends to be a young mans game), and they’re very stiff. Agility comes from keeping the weight down and being prepared to act quickly. This means keeping things simple, ensuring that we have minimum set of moving parts required. They might have an eye for detail, such as nitrogen in the tyres, but unnecessary moving parts that might reduce reliability or performance are eliminated.

This gold plated approach to SOA creates a lot of unrequired flexibility, this additional flexibility increases complexity, and the complexity becomes the boat anchor that slows you down and stops you from being agile. Turning the car is no longer a simple of tugging on the steering wheel, as we need governance to stop us from pulling the wrong lever in the bank of 500 identical levers in front of us.

It's really that simple!
It's really that simple!

We’ve made everything too complicated. Mario was wrong: too much is too much.

What we need is some guidance – a way of scoping and directing the flexibility we’re going to create. Governance isn’t enough, as governance is focused on stopping bad things from happening. We have a scoping problem. Our challenge is to understand what flexibility will be required in the future, and agreeing on the best way to support it.

To date I’ve been using a very fuzzy “business interest” metric for this, where services are decomposed until the business is no longer interested. The rational is that we put the flexibility only were the business thinks it needs to focus. This approach works fairly well, but it relies too much on the tacit judgement of a few skilled business analysts and architects, making it too opaque and hard to understand for the people not involved in the decision making process. It’s also hard to scale. We need something more deterministic and repeatable.

Which brings me to a friend’s MBA thesis, which he passed to me the other week. It’s an interesting approach to building business cases for IT solutions, one based on real options.

The problem with the usual approaches to building a business case, using tools like net present value (NPV) and discounted cash flow, is that we assume that the world doesn’t change post the decision to build the solution (or not). They don’t factor in the need to change a solution once it’s in the field, or even during development.

The world doesn’t work this way: the solution you approved in yesterday’s business environment will be deployed into a radically different business environment tomorrow. This makes it hard to justify the additional investment required for a more flexible SOA based solution, when compared to a conventional monolithic solution. The business case doesn’t include flexibility as a factor, so more flexible (and therefore complex and expensive) solutions lose to the cheaper, monolithic approach.

Real options address this by pushing you down a scenario planning based approach. You estimate the future events that you want to guard against, and their probabilities, creating a set of possible futures. Each event presents you with options to take action. The action, for example, might be to change, update or replace components in the solution to bring them in line with evolving business realities. The options are – in effect – flex-points that we might design into our solutions SOA. The real options methodology enables us to ascribe costs to these future events and the create a decision tree that captures the benefits of investing in specific flex points, all in a clear and easily understandable chain of reasoning.

The decision tree and options provide us with a way to map out where to place flex points in the SOA solution. They also provide us with strong guidance on how much flexibility to introduce. And this is the part I found really interesting about the approach. It also provides us with a nice framework to govern the evolution of the SOA solution, as changes are (generally) only made when an option is taken: when it’s business case is triggered.

It’s a bit like those formula one cars. A friend of mine used to work for one F1 manufacturer designing and testing camshafts. These camshafts had to fall within a 100,000 lifetime revolution window. An over-designed camshaft was unnecessary weight, while an under-designed one means that you wouldn’t win (or possibly even finish) the race. Work it out: a 100,000 revolutions is a tiny window for an F1 car, given the length of a race.

An approach like real options helps us ensure that we only have the flexibility required in the solution, and that it is exactly where it is required. Not too much, and not too little. Just enough to help us win the race.

Innovation [2009-10-05]

Another week and another collection of interesting ideas from around the internet.

As always, thoughts and/or comments are greatly appreciated.

This issue:

Accelerate along the road to happiness

Our ability to effectively manage time is central to success in today’s hype-competitive business environment. The streamlined and high velocity value-chains we’ve created are designed to invest as little time (and money) as possible in unproductive business activities. However, being fast, being good at optimizing our day-to-day operations, is no longer enough. We’ve reached a point where managing the acceleration of our business—the ability to change direction, redeploy resources to meet new opportunities more rapidly than our competition—is the driver for best in category performance. If we can react faster than our competition then we can capitalize on a business opportunity (or disruption, as they are often the same) and harvest any value the opportunity created.

Time is our overarching business driver at the moment. We hope to be the first to approve a mortgage, capturing the customer before our competitors have even responded to the original application. We strive to be first to market with a new portable music device (Walkman or iPod), establishing early mover advantage and taking the dominant position in the market. Or we might simply want to quickly restore essential services—power, gas or water—to our customers, as they have become intensely dependent upon them. Globalization has leveled the playing field, as we’re all working from the same play book and leveraging the same resources. The most significant factor for success in this environment is the ability to execute faster than our competition—harvesting the value in an opportunity before they can.

This focus on time is a recent phenomena. Not long ago, no further back than the early nineties, we were more concerned with mass. The challenge was too get the job done. Keep the wheels turning in the factories. Keep the workers busy in their cubicles. Time is money, so we’re told, and we need to ensure that we don’t waste money by laying idle. Mass was the key to success—ensuring that we had enough work to do, enough raw materials to work on, to keep our business busy and productive.

When mass is the focus, then bigger is better. This is a world where global conglomerates rule, as size is the driver for success. Supply chains were designed so that enough stuff was available right next to the factory, where supply can be ensured, that the factory would never run out of raw materials and grind to a halt. Whether shuffling paperwork or shifting widgets, the ability to move more stuff around the business was always seen as an improvement.

This is also the world that created a pile of shipping containers too behold in the Persian Gulf, during the Gulf War in the early nineties. With no known destination, some containers couldn’t be delivered. Without a clear understanding of where they came from, others couldn’t be returned. A few of these orphaned containers were opened in an attempt to determine their destination or origin; however the sweltering Arabian sun was not kind to their contents, which included items such as raw poultry, so a stop was soon put to that. The containers just kept piling up. 22,000 of 50,000 containers simply became invisible, collecting in a pile that went by the jaunty name of Iron Mountain.

Iron Mountain: 22,000 containers that became invisible
Iron Mountain: 22,000 containers that became invisible

Our answer was to stop focusing on mass, on having enough stuff on hand to keep the wheels of industry turning. We have to admit that Iron Mountain proves that we could move sufficient mass. The next challenge was to ensure that materials arrived at just the right time for them to be consumed by the business. We moved from worrying about mass, to managing velocity.

Total quality management and process improvement efforts finally found their niche. LEAN and Six Sigma rolled through the business landscape ripping cost out businesses where-ever they went. Equipped with books on Toyota’s Production System and kanban cards, we ripped excess material from the supply chain. Raw materials arrive just-in-time, and we avoid the costs associated with storing and handling vast warehouses of material, as well as the working capital tied up in the stored material itself. Quality went up, process cycle times shrunk, and the pace of business accelerated. Much like the tea clippers from China in the 1800s, with the annual race to get the first crop back to London for the maximum profit (with skipper paid a profit share as an incentive along with their salary), we’re focused on cranking the handle of business as fast as possible.

Zara, a fashion retailer, is the poster child for this generation of business. The fashion industry is built around a value-chain that tries to push out regular product updates, beating up demand via runway shows and media coverage to support a seasonal marketing cycle. Zara takes a different approach, tracking customer preferences and trends as they happen in the stores and trying to deliver an appropriate design as rapidly as possible, allowing customer demand to pull fashion. By focusing on responding to customer demand, wherever it is, Zara has built an organization designed too minimize time from design to marketed product. For example, onshore, high-tech, agile production is preferred to low-tech but low cost, offshore production which involves long production delays. Zara takes two weeks to take a product to market, where the industry average is six months; the lifetime of Zara’s products is measured in weeks, rather than months; and the products offered in each store are tailored to the interests of the community it serves rather than a long term marketing plan.

The change in product life-cycle has created a material change to customer buying habits. Traditionally customers’ will visit a fashion store a few times a year to see what a new season brings. There is no real pressure to buy in any particular visit, as they know they can return to buy the same garment later. Zara, however, with it’s dramatically shortened product cycles, drives different behavior. Customer visit more often, as they can expect to see a new range each visit. They are also more likely too buy, as they know that there is little chance of the same garment being available the next time. This approach has made Zara the most profitable arm of Inditex, a holding company of eight retail brands, and one of the biggest success stories in Spanish business.

The dirty secret of high velocity, lean businesses is that they are fragile: small disturbances can create massive knock-on effects. As we’ve ripped fat from the value chain, we’ve also weakened its ability to react to, and resolve, disruptions. A stockout can now flow all the way back along the supply chain to the literal coal face, stalling the entire business value-chain. Restoring an essential service is delayed while we scramble to procure the vital missing part. Mortgage approvals are deferred while we try reallocate the work load of a valuer dealing with a personal emergency. Or our carefully synchronized product launch falls apart for what seems like a trivial reason somewhere on the other side of the globe.

Our most powerful tools in creating todays high velocity businesses—tools like straight-through processing, LEAN and Six Sigma—worked by removing variation from business processes to increase throughput. The same tools prevent us from effectively responding to these disruptions.

Opportunities today are more frequent, but disruptive and fleeting. An open air festival in the country might represent an opportunity for a tolling operator to manage parking in an adjacent field, if the solution can be deployed as sufficient scale rapidly enough. Or the current trend for pop-up retail stores (if new products rapidly come and go, then why not stores) could be moved from an exceptional, special occasion marketing tool, into the mainstream as a means to optimize sales day-by-day. Responding to these opportunities implies reconfiguring our business on the fly—rapidly integrating business exceptions into the core of our business. This might range from reconfiguring our carefully designed global supply chain, through changing core mortgage approval criteria and processes to modifying category management strategies in (near) real time.

Sam: Waiting while his bank sorts itself out
Sam: Waiting while his bank sorts itself out

We’re entering a time when our ability to change direction, adapting to and leveraging changes in the commercial environment as they occur, will drive our success. If we can react faster than the competition then we can capitalize on a business opportunity and harvest any value the opportunity creates. Our focus will become acceleration: working too build businesses with the flexibility and spare energy required to turn and respond rapidly. These businesses will be the F1 cars of business, providing a massive step in performance over more conventional organizations. And, just like F1, they will also require a new level of performance from our knowledge workers. If acceleration is our focus, then our biggest challenge will be creating time and space required by our knowledge workers to identify these opportunities, turn the steering wheel and leverage them as they occur.

Update: A friend of mine just pointed out that the logical progression of mass → velocity → acceleration naturally leads to jerk, which is an informal unit of measurement for the third derivative.