Tag Archives: software development

The IT department we have today is not the IT department we’ll need tomorrow

The IT departments many of us work in today (either as an employee or consultant) are often the result of thirty or more years of diligent labour. These departments are designed, optimised even, to create IT estates populated with large, expensive applications. Unfortunately these departments are also looking a lot like dinosaurs: large, slow and altogether unsuited for the the new normal. The challenge is to reconfigure our departments, transforming them from asset management functions into business (or business-technology) optimisation engines. This transformation should be a keen interest for all of us, as it’s going to drive a dramatic change in staffing profiles which will, in turn, effect our own jobs in the no so distant future.

Delivering large IT solutions is a tricky business. They’re big. They’re expensive. And the projects to create them go off the rails more often than we’d like to admit. IT departments have been built to minimise the risks associated with delivering and operating these applications. This means governance, and usually quite a lot of it. Departments which started off as small scale engineering functions soon picked up an administrative layer responsible to the mechanics of governance.

More recently we’ve been confronted with the challenge with managing the dependancies and interactions between IT applications. Initiatives like straight-through processing require us to take a holistic, rather than a pieces-parts, approach, and we’re all dealing with the problem of having one of each application or middleware product, as well as a few we brewed in the back room ourselves. Planning the operation and evolution of the IT estate became more important, and we picked up an enterprise architecture capability to manage the evolution of our IT estate.

It’s common to visualise these various departmental functions and roles as a triangle (or a pyramid, if you prefer). At the bottom we have engineering: the developers and other technical personnel who do the actual work to build and maintain our applications. Next layer up is governance, the project and operational administrators who schedule the work and check that it’s done to spec. Second from the top are the planners, the architects responsible for shaping the work to be done as well as acting as design authority. Capping of the triangle (or pyramid) is the IT leadership team who decide what should be done.

The departmental skills triangle

While specific techniques and technologies might come and go, the overall composition of the triangle has remained the same. From the sixties and seventies through to even quite recently, we’ve staffed our IT departments with many technical doers, a few less administrators, a smaller planning team, and a small IT leadership group. The career path for most of us been a progression from the bottom layers – when we were fresh out of school – to the highest point in the triangle that we can manage.

The emergence of off-shore and outsourcing put a spanner in the works. We all understand the rational: migrate the more junior positions – the positions with the least direct (if any) contact with the business proper – to a cheaper country. Many companies under intense cost pressure broke the triangle in two, keeping the upper planning and decision roles, while pushing the majority of the manage and all the do roles out of the country, or even out of the company.

Our first attempt at out-sourcing

Ignoring whether or not this drive to externalise the lower roles provided the expected savings or not, what it did do is break the career ladder for IT staff. Where does you next generation of senior IT personnel come from if you’ve pushed the lower ranks out of the business? Many companies found themselves with an awkward skills shortage a few years into an outsourcing / off-shore arrangement, as they were no longer able to train or promote senior personnel to replace those who were leaving through natural attrition.

The solution to this was to change how we brake-up the skills triangle; rather than a simple horizontal cut, we took a slice down the side. Retaining a portion of all skills in-house allows companies provide a career path and on the job training for their staff.

A second, improved, go at out-sourcing
A second, improved, go at out-sourcing

Many companies have tweaked this model, adding a bulge in the middle to provide a large enough resource pool to manage both internal projects, as well as those run by out-sourced and off-shore resources.

Factoring in the effort required to manage out-sourced projects
Factoring in the effort required to manage out-sourced projects

This model is now common in a lot of large companies, and it has served us well. However, the world has a funny habit of changing just when you’ve everything working smoothly.

The recent global financial criss has fundamentally changed the business landscape. We are experiencing not merely another turn of the business cycle, but a restructuring of the economic order. Many are even talking about the emergence of a new normal. The impact this will have on how we run our businesses (and our IT departments) is still being discussed, but we can see the outline of this impact already.

Companies are becoming more focused, while leaning more heavily on partners and services companies (BPO, out-sourcers, consultants, and so on) to cover those areas of the business they don’t want to focus on. We can see this from the global companies who have effectively moved to a franchise model, though to the small end of town where startups are using on-line services such as Amazon S3, rather than building internal capabilities. While this trend might have initially started as a cost saving, most of the benefit is in management time saved, which can then be used to focus on more important issues. We’re all finding that the limiting factor in our business is management time, so being able to hand off the management of less important tasks can help provide that edge you need.

We’re also seeing faster business change: what used to take years now takes months, or even weeks. The constant value-chain optimisation we’ve been working on since the 70s has finally cumulated in product and regulatory life-cycles that change faster than we can keep up. Nowhere is this more evident than the regulated industries (finance, utilities …), where updates in government regulation has changed from a generational to a quarterly occurrence as governments attempt to use regulation change to steer the economic boat.

Money is also becoming (or has become) more expensive, causing companies and deals to operate with less leverage. This means that there is less capital available for major projects, pushing companies to favour renting over buying, as well as creating a preference for smaller, incremental change over the major business transformation of the past.

And finally, companies are starting to take a truly global outlook and operate as one cohesive business across the globe, rather than as a family of cloned business who operate more-or-less independently in each region.

We can draw a few general conclusions on the potential impact on IT departments of these trends.

  • The increase reliance on partners, the broader partner ecosystem this implies, and an increasingly global approach to business will create more complex operational environments, increasing the importance of planning the IT estate and steering a company’s IT in the right direction.
  • The need to reduce leverage, and free up working capital, is pushing companies toward BPO and SaaS solutions, rather than the traditional on-premisses solutions, where the solution provider is paid per-seat, or might even be only paid a success fee.
  • The need for rapid project turn-around is pushing us toward running large portfolios of small projects, rather than a small number of large projects.
  • A lot of the admin work we used to do is now baked into web delivered solutions (BaseCamp et al).

This will trigger us to break up a the skills triangle in a different way.

A skills/roles triangle for the new normal
A skills/roles triangle for the new normal

While we’ll still take a slice down the side of the triangle, the buldge will move to the ends of the slice, giving it a skinny waist. The more complex operational environment means that we need to beef up planning (though we don’t want to get all dogmatic about our approach, as existing asset-centric IT planning methodologies won’t work in the new normal). A shift to large numbers of small projects (where the projects are potentially more technically complex) means that we’ll beef up our internal delivery capability, providing team leads with more autonomy. The move to smaller projects also means that we can reduce our administration and governance overhead.

We’ll replace some skills with automated (SaaS) solutions. Tools like BaseCamp will enable us to devolve responsibility for reporting and management to the team at the coalface. It will also reduce the need to develop and maintain infrastructure. Cloud technology is a good example of this, as it takes a lot of the tacit knowledge required to manage a fleet of servers and bakes it into software, placing it in the hands of the developers. Rumor has it that that a cloud admin can support 10,000 servers to a more traditional admin’s 500.

And finally, our suppliers act as a layer through the middle, a flex resource for us to call on. They can also provide us with a broader, cross-industry view, of how to best leverage technology.

This thinning out of the middle ranks is part of a trend we’re seeing elsewhere. Web2.0/E2.0/et al are causing organisations to remove knowledge workers — the traditional white collar middle layers of the organisaiton – leaving companies with a strategy/leadership group and task workers.

Update: Andy Mulholland has an interesting build on this post over at the Capgemini CTO blog. I particularly like the Holm service launched by Ford and Microsoft, a service that it’s hard to imagine a traditional IT department fielding.

Consulting doesn’t work any more. We need to reinvent it.

What does it mean to be in consulting these days? The consulting model that’s evolved over the last 30 – 50 years seems to be breaking down. The internet and social media have shifted the way business operates, and the consulting industry has failed to move with it. The old tricks that the industry has relied on — the did it, done it stories and the assumption that I know something you don’t — no longer apply. Margins are under pressure and revenue is on the way down (though outsourcing is propping up some) as clients find smarter ways to solve problems, or decide that they can simply do without. The knowledge and resources the consulting industry has been selling are no longer scarce, and we need to sell something else. Rather than seeing this as a problem, I see it as a huge opportunity; an opportunity to establish a more collaborative and productive relationship founded on shared, long term success. Sell outcomes, not scarcity and rationing.

I’m a consultant. I have been for some time too, working in both small and large consultancies. It seems to me that the traditional relationship between consultancy and client is breaking down. This also appears to be true for both flavours of consulting: business and technology. And by consulting I mean everything from the large tier ones down to the brave individuals carving a path for themselves.

Business is down, and the magic number seems to be roughly a 17% decline year-on-year. One possible cause might be that the life blood of the industry — the large multi-year transformation project — has lost a lot of its attraction in recent years. If you dig around in the financials for the large publicly listed consultancies and vendors you’ll find that the revenue from IT estate renewal and transformation (application licenses, application configuration and installation services, change management, and even advisory) is sagging by roughly 17% everywhere around the globe.

SABER @ American Airlines

Large transformation projects have lost much of their attraction. While IBM successfully delivered SABER back in the 60s, providing a heart transplant for American Airlines ticketing processes, more recent stabs at similarly sized projects have met with less than stellar results. Many more projects are quietly swept under the carpet, declared a success so that involved can move on to something else.

The consulting model is a simple one. Consultants work on projects, and the projects translate into billable hours. Consultancies strive to minimise overheads (working on customer premises and minimising support staff), while passing incidental costs through to clients in the form of expenses. Billable hours drive revenue, with lower grades provide higher margins.

This creates a couple of interesting, and predictable, behaviours. First, productivity enhancing tooling is frowned on. It’s better to deploy a graduate with a spreadsheet than a more senior consultant with effective tooling. Second, a small number of large transactions are preferred to a large number of small transactions. A small number of large transactions requires less overhead (sales and back-office infrastructure).

All this drives consultancies to create large, transformational projects. Advisory projects end up developing multi-year (or even multi-decade) roadmaps to consolidate, align and optimise the business. Technology projects deliver large, multi-million dollar, IT assets into the IT estate. These large, business and IT transformation projects provide the growth, revenue and margin targets required to beat the market.

This desire for large projects is packaged up in what is commonly called “best practice”. The consulting industry focuses on did it, done it stories, standard and repeatable projects to minimise risk. The sales pitch is straight-forward: “Do you want this thing we did over here?” This might be the development of a global sourcing strategy, an ERP implementation, …

Spencer Tracy & Katharine Hepburn in The Desk Set
Spencer Tracy & Katharine Hepburn in The Desk Set

This approach has worked for some time, with consultancy and client more-or-less aligned. Back when IBM developed SABER you were forced to build solutions from the tin up, and even small business solutions required significant effort to deliver. In the 1957, when Spencer Tracy played a productivity expert in The Desk Set, new IT solutions required very specific skills sets to develop and deploy. These skills were in short supply, making it hard for an organisation to create and maintain a critical mass of in-house expertise.

Rather than attempt to build an internal capability — forcing the organisation on a long learning journey, a journey involving making mistakes to acquire tacit knowledge — a more pragmatic approach is to rent the capability. Using a consultancy provides access to skills and knowledge you can’t get elsewhere, usually packaged up as a formal methodology. It’s a risk management exercise: you get a consultancy to deliver a solution or develop a strategy as they just did one last week and know where all the potholes are. If we were cheeky, then we would summerize this by stating that consultancies have a simple value proposition: I know something you don’t!

It’s a model defined by scarcity.

A lot has changed in the last few years; business moves a lot faster and a new generation of technology is starting to take hold. The business and technology environment is changing so fast that we’re struggling to keep up. Technology and business have become so interwoven that we now talk of Business-Technology, and a lot of that scarce knowledge is now easily obtainable.

The Diverging Pulse Rates of Business and Technology
The Diverging Pulse Rates of Business and Technology

The scarce tacit knowledge we used to require is now bundled up in methodologies; methodologies which are trainable, learnable, and scaleable. LEAN and Six Sigma are good examples of this, starting as more black art than science, maturing into respected methodologies, to today where certification is widely available and each methodology has a vibrate community of practitioners spread across both clients and consultancies. The growth of MBA programmes also ensures that this knowledge is spread far and wide.

Technology has followed a similar path, with the detailed knowledge required to develop distributed solutions incrementally reified in methodologies and frameworks. When I started my career XDR and sockets were the networking technologies of the day, and teams often grew to close to one hundred engineers. Today the same solution developed on a modern platform (Java, Ruby, Python …) has a team in the single digits, and takes a fraction of the time. Tacit knowledge has be reified in software platforms and frameworks. SaaS (Software as a Service) takes this to a while new level by enabling you to avoid software development entirely.

The did it, done it stories that consulting has thrived on in the past are being chewed up and spat out by the business schools, open source, and the platform and SaaS vendors. A casual survey of the market usually finds that SaaS-based solutions require 10% of the installation effort of a traditional on-premsis solution. (Yes, that’s 90% less effort.) Less effort means less revenue for the consultancies. It also reduces the need for advisory services, as provisioning a SaaS solution with the corporate credit card should not require a $200,000 project to build a cost-benefit analysis. And gone are the days when you could simply read the latest magazines and articles from the business schools, spouting what you’d read back to a client. Many clients have been on the consulting side of the fence, have a similar education in the business schools, and reads all the same articles.

I know and you don’t! no longer works. The world has moved on and the consulting industry needs to adapt. The knowledge and resources the industry has been selling are no longer scarce, and we need to sell something else. I see this is a huge opportunity; an opportunity to establish a more collaborative and productive relationship founded on shared, long term success. As Jeff Jarvis has said: stop selling scarcity, sell outcomes.

Updated: A good friend has pointed out the one area of consulting — one which we might call applied business consulting — resists the trend to be commoditized. This is the old school task of sitting with clients one-on-one, working to understand their enterprise and what makes it special, and then using this understanding to find the next area or opportunity that the enterprise is uniquely qualified to exploit. There’s no junior consultants in this area, only old grey-beards who are too expensive to stay in their old jobs, but that still are highly useful to the industry. Unfortunately this model doesn’t scale, forcing most (if not all) consultancies into a more operational knowledge transfer role (think Six Sigma and LEAN) in an attempt to improve revenue and GOP.

Updated: Keith Coleman (global head of public sector at Capgemini Consulting) makes a similar case with Time to sell results, not just advice (via @rpetal27).

Updated: I’ve responded to my own post, tweaking my consulting page to capture my take on what a consultant needs to do in this day and age.

Why we can’t keep up

We’re struggling to keep up. The pace of business seems to be constantly accelerating. Requirements don’t just slip anymore: they can change completely during the delivery of a solution. And the application we spent the last year nudging over the line into production became instant legacy before we’d even finished. We know intuitively that only a fraction of the benefits written into the business case will be realized. What do we need to do to get back on top of this situation?

We used to operate in a world where applications were delivered on time and on budget. One where the final solution provided a demonstrable competitive advantage to the business. Like SABER, and airline reservation system developed for American Airlines by IBM which was so successful that the rest of the industry was forced to deploy similar solutions (which IBM kindly offered to develop) in response. Or Walmart, who used a data warehouse to drive category leading supply chain excellence, which they leveraged to become the largest retailer in the world. Both of these solutions were billion dollar investments in todays money.

The applications we’ve delivered have revolutionized information distribution both within and between organizations. The wave of data warehouse deployments triggered by Walmart’s success formed the backbone for category management. By providing suppliers with a direct feed from the data warehouse—a view of supply chain state all the way from the factory through to the tills—retailers were able to hand responsibility for transport, shelf-stacking, pricing and even store layout for a product category to their suppliers, resulting in a double digit rises in sales figures.

This ability to rapidly see and act on information has accelerated the pulse of business. What used to take years now takes months. New tools such as Web 2.0 and pervasive mobile communications are starting to convert these months into week.

Take the movie industry for example. Back before the rise of the Internet even bad films could expect a fair run at the box-office, given a star billing and strong PR campaign too attract the punters. However, post Internet, SMS and Twitter, the bad reviews have started flying into punters hands moments after the first screening of a film has started, transmitted directly from the first audience. Where the studios could rely a month or of strong returns, now that run might only last hours.

To compensate, the studios are changing how they take films to market; running more intensive PR campaigns for their lesser offerings, clamping down on leaks, and hoping to make enough money to turn a small profit before word of mouth kicks in. Films are launched, distributed and released to DVD (or even iTunes) in weeks rather than months or years, and studios’ funding, operations and the distribution models are being reconfigured to support the accelerated pace of business.

While the pulse of business has accelerated, enterprise technology’s pulse rate seems to have barely moved. The significant gains we’ve made in technology and methodologies has been traded for the ability to build increasingly complex solutions, the latest being ERP (enterprise resource planning) whose installation in a business is often compared to open heart surgery.

The Diverging Pulse Rates of Business and Technology

This disconnect between the pulse rates of business and enterprise technology is the source of our struggle. John Boyd found his way to the crux of the problem with his work on fighter tactics.

John Boyd—also know as “40 second Boyd”—was a rather interesting bloke. He had a standing bet for 40 dollars that he beat any opponent within 40 seconds in a dog fight. Boyd never lost his bet.

The key to Boyd’s unblemished record was a single insight: that success in rapidly changing environment depends on your ability to orient yourself, decide on, and execute a course of action, faster than the environment (or your competition) is changing. He used his understanding of the current environment—the relative positions, speed and performance envelopes of both planes—to quickly orient himself then select and act on a tactic. By repeatedly taking decisive action faster than his opponent can react, John Boyd’s actions were confusing and unpredictable to his opponent.

We often find ourselves on the back foot, reacting to seemingly chaotic business environment. To overcome this we need to increase the pulse of IT so that we’re operating at a higher pace than the business we support. Tools like LEAN software development have provided us with a partial solution, accelerating the pulse of writing software, but if we want to overcome this challenge then we need to find a new approach to managing IT.

Business, however, doesn’t have a single pulse. Pulse rate varies by industry. It also varies within a business. Back office compliance runs at a slow rate, changing over years as reporting and regulation requirements slowly evolve. Process improvement and operational excellence programs evolve business processes over months or quarters to drive cost out of the business. While customer or knowledge worker facing functionality changes rapidly, possibly even weekly, in response to consumer, marketing or workforce demands.

Aligning technology with business

We can manage each of these pulses separately. Rather than using a single approach to managing technology and treating all business drivers as equals, we can segment the business and select management strategies to match the pulse rate and amplitude of each.

Sales, for example, is often victim of an over zealous CRM (customer relationship management) deployment. In an effort to improve sales performance we’ll decide to role out the latest-greatest CRM solution. The one with the Web 2.0 features and funky cross-sell, up-sell module.

Only of a fraction of the functionality in the new CRM solution is actually new though—the remainder being no different to the existing solution. The need to support 100% of the investment on the benefits provided by a small fraction of the solution’s features dilutes the business case. Soon we find ourselves on the same old roller-coaster ride, with delivery running late,  scope creeping up, the promised benefits becoming more intangible every minute, and we’re struggling to keep up.

There might be an easier way. Take the drugs industry for example. Sales are based on relationships and made via personal calls on doctors. Sales performance is driven by the number of sales calls a representative can manage in a week, and the ability to answer all of a doctor’s questions during a visit (and avoid the need for a follow-up visit to close the sale). It’s not uncommon for tasks unrelated to CRM—simple tasks such as returning to the office to process expenses or find an answer to a question—to consume a disproportionate amount of time. Time that would be better spent closing sales.

One company came up with an interesting approach. To support the sales reps in the field they provided them with the ability to query the team back in the office, answering a clients question without the need to return to head office and then try to get back in their calendar. The solution was to deploy a corporate version of Twitter, connecting the sales rep into the with the call center and all staff using the company portal via a simple text message.

By separating concerns in this way—by managing each appropriately—we can ensure that we are working at a faster pace than the business driver we supporting. By allocating our resources wisely we can set the amplitude of each pulse. Careful management of the cycles will enable us to bring business and technology into alignment.

Managing technology, not applications

We’re getting it all wrong—we focused on managing the technology delivery process rather than the technology itself. Where do business process outsourcing (BPO), software as a service (SaaS), Web 2.0 and partner organisations sit in our IT strategy? All too often we focus on the delivery of large IT assets into our enterprise, missing the opportunity to leverage leaner disruptive solutions that could provide a significantly better outcome for the business.

IT departments are, by tradition, inward looking asset management functions. Initially this was a response to the huge investment and effort required to operate early mainframe computers, while more recently it has been driven by the effort required to develop and maintain increasingly complex enterprise applications. We’ve organised our IT departments around the activities we see as key to being a successful asset manager: business analysis, software development & integration, infrastructure & facilities, and project or programme management. The result is a generation of IT departments closely aligned with the enterprise application development value-chain, as we focus on managing the delivery of large IT assets into the enterprise.

Building our IT departments as enterprise application factories has been very successful, but the maturation of applications over the last decade and recent emergence of approaches like SaaS means that it has some distinct limitations today. An IT department that defines itself in terms of managing the delivery of large technology assets tends to see a large technology asset as the solution to every problem. Want to support a new pricing strategy? Need to improve cross-sell and up-sell? Looking for ways to support the sales force while in the field? Upgrade to the latest and greatest CRM solution from your vendor of choice. The investment required is grossly out of proportion with the business benefit it will bring, making it difficult to engage with the rest of the business who view IT as a cost centre rather than an enabler.



A typical IT department value-chain

Unfortunately the structure of many of our IT departments—optimised to create large IT assets—actively prohibits any other approach. More incremental or organic approaches to meeting business needs are stopped before they even get started, killed by an organisation structure and processes that impose more overhead than they can tolerate.

Applications were rare and expensive during most of enterprise IT’s history, but today they are plentiful and (comparativly) cheap. Software as a Service (SaaS) is also emerging to provide best of breed functionality but with a utillity delivery model; leveraging an externally managed service and paying per use, rather requiring capital investment in an IT asset to provide the service internally. Our focus is increasingly turned to ensuring that business processes and activities are supported with an appropriate level of technology, leveraging solutions from traditional enterprise applications through to SaaS, outsourced solutions or even bespoke elements where we see fit. We need to be focused on managing technology enablement, rather than IT assets, and many IT departments are responding to this by reorganising their operations to explore new strategies for managing IT.

Central to this new generation of IT departments is a sound understanding of how the business needs to operate—what it wants to be famous for. The old technology centric departmental roles are being deprecated, replaced with business centric roles. One strategy is to focus on Operational Excellence, Technology Enablement and Contract Management. A number of Chief Process Officer (CPO) roles are created as part of the Operational Excellence team, each focusing on optimising one or more end-to-end processes. The role is defined and measured by the business outcomes it will deliver rather than by the technology delivery process. CPOs are also integrating themselves with organisation wide business improvement and operational excellence initiatives, taking a proactive stance with the business instead of reactively waiting for the business to identify a need.



Managing technology, not applications

The Technology Enablement team works with Operational Excellence to deliver the right level of technology required to support the business. Where Operational Excellence looks out into the business to gain a better understanding of how the business functions, Technology Enablement looks out into the technology community to understand what technologies and approaches can be leveraged to create the most suitable solution. (As opposed to traditional, inward focused IT department concerned with developing and managing IT assets.) These solutions can range from SaaS through to BPO, AM (application management), custom development or traditional on-premises applications. However, the mix of solutions used will change over time as we move from today’s application centric enterprise IT to new process driven approaches. Solutions today are dominated by enterprise applications (most likely via BPO or AM), but increasingly shifting to utility models such as SaaS as these offerings mature.

Finally a contract management team is responsible for managing the contractual & financial obligations, and service level agreements between the organisation and suppliers.

One pronounced effect of a strongly business focused IT organisation is the externalisation of many asset management activities. Rather than trying to be good at everything needed to deliver a world class IT estate, and ending up beginning good at nothing, the department focuses its energies on only those activities that will have the greatest impact on the business. Other activities are supported by a broad partner ecosystem: systems integrators to install applications, outsourcers for application management and business process outsourcing, and so on. Rather than ramping up for a once-in-four-year application renewal—an infrequent task for which the department has trouble retaining expertise—the partner ecosystem ensures that the IT department has access to organisations whose core focus is installing and running applications, and have been solving this problem every year for the last four years.

This approach allows the IT department to concentrate on what really matters for the business to succeed. Its focus and expertise is firmly on the activities that will have the greatest impact on the business, while a broad partner ecosystem provides world class support for the activities that it cannot afford to develop world class expertise in. Rather than representing a cost centre in the business, the IT department can be seen as an enabler, working with other business to leverage new ideas and capabilities and drive the enterprise forward.