ABOUT THE API

The Advanced Performance Institute (API) is the world’s leading independent research and advisory organization specializing in organizational performance. The institute provides expert knowledge, research, consulting and training on concepts such as Strategic Performance Management, Performance Measurement and Enterprise Business Intelligence.

The aim of the API is to provide today’s performance focused organizations with insights, advice and services that help them deliver superior performance. Customers of the API are wide ranging and include many of the world’s leading blue chip companies as well as public sector organizations, governments and not-for-profit organizations around the globe.

Some of the services offered by API are summarized below.

Knowledge and research

• The API conducts internationally recognized research with the aim of understanding and sharing the latest trends and best practices in the field of managing, measuring and analysing organizational performance.

• A wide selection of case studies, research reports, articles and management white papers are freely available to download from the API website.

Audit and reviews

• Extensive research and implementation experience across the world puts the API into a perfect position to assess existing performance management and business intelligence approaches and compare them with current global best practice.

• Audit and benchmarking solutions help organizations identify where they can improve their performance and get more value from their performance management and BI initiatives.

Consulting

• Based on the latest thinking, the institute can deliver a proven and tested process for designing performance management frameworks and initiatives.

• Perfected through real-life implementation experience across many industries, the institute is able to facilitate each step of the design process to ensure clearly articulated strategies, state of the art strategic maps, meaningful performance indicators and aligned processes, so that performance information is communicated and used to inform day-to-day decision making and learning.

Training

• The API provides training and coaching on any issues related to performance management and business intelligence. These are offered as either open enrolment training courses, or for maximum impact, can be delivered in customized workshops and training sessions within organizations.

For more information, and to download case studies and articles, please visit: www.ap-institute.com.

FOREWORD

Bernard Marr’s book on The Intelligent Company and evidence-based management at times draws an apt comparison between business and medicine. The use of evidence as the primary guide to decisions and actions in business is clearly a similar idea to ‘evidence-based medicine.’ Yet the idea of using evidence in business decisions may elicit the same reaction I had when I first heard of evidence-based medicine. I thought, “Evidence-based medicine sounds great, but just what was medicine based on in the past?”

Just as I thought (incorrectly, as it happens) that science, medical evidence, and data were the primary basis of clinical practice, you may hold the comforting belief that evidence and data are widely used in making business decisions. Alas, that is often not the case. Despite the rapidly-growing availability of information from online transaction systems, the internet, point-of-sale systems, and other sources, our use of evidence-based management isn’t growing at the same pace. Many managers still manage and decide as if modern computing tools didn’t exist.

There are several facets to this problem. First, there is still too much use of pure intuition in business. Intuition has its place in decision-making, particularly when it’s based on experience refined over time. But that place should generally be last, not first. If you can’t get data, if you can’t do an experiment, and if you can’t access the stored knowledge of your organization, then by all means use your intuition. But explore those other decision resources first.

Secondly, there is just too weak a link between the information that organizations do have available and the decisions that they make. As with medicine, some of my very intelligent friends who don’t work in businesses often assume that the prevalence of ‘business intelligence’ systems and analytics of various types has to mean better business decision-making. But this assumption is often violated. Business intelligence systems generate reports that don’t yield any decision or any action. IT organizations develop data warehouses with little notion of how they will be applied to decisions. Analytics are generated, but executives ignore them in favor of intuition or comfortable past practices. Most firms don’t know what their most important decisions are, and how technology and information might be used to inform them.

The good news is that this regrettable situation is slowly improving. My own work on analytical competition suggests that some firms have made a strategic capability out of their data and analytical capabilities. In other firms, executives are beginning to look around at their massive investments in information systems and saying, “Weren’t we supposed to be able to run the business better as a result of these?” Many are working to improve particular decision processes; in a recent study I did of 57 companies, about 90% could identify at least one important decision that they had tried to improve with data and analysis. Frequently-made decisions, such as product pricing or loan and policy origination in financial services companies, are increasingly being automated, with analytical or rule-based criteria embedded into business processes. There is, as Marr suggests, a long way to go, but progress toward evidence-based management is clearly being made.

Marr’s book will clearly be of major benefit to individuals and organizations wanting to address the problem of insufficient evidence-based management. It fills a gap in the marketplace of ideas, in that no previous book has combined both a structured orientation to fact-based decisions, and a focus on using the outputs of business intelligence systems to make those decisions. While the book is all about the value of data and analysis, even the most mathematics-shy executive will not be deterred by the terminology and writing. And it considers not only the process and information for evidence-based management, but also the cultural and organizational changes necessary to bring it about.

Marr correctly compares the process of business decision-making to the scientific method. The approaches to analyzing and deciding scientific data will eventually be adopted in most or all successful businesses. It may take a while, but the paragons of evidence-based management already exist, and you will read about them in this book. However, if you adopt these approaches today in a widespread and diligent fashion, you will still be able to derive competitive advantage from their use.

Thomas H. Davenport
President’s Distinguished Professor
in Information Technology and Management,
Babson College, Massachusetts, USA

ACKNOWLEDGEMENTS

It is impossible to write a book in isolation and therefore I need to thank everyone who has influenced my thinking on The Intelligent Company. In particular, my role in heading up the Advanced Performance Institute provides me with all the opportunities I need to develop and test new ideas, and I am indebted to my colleagues and the current Fellows of the Advanced Performance Institute: David Teece, Rob Austin, Dean Spitzer, Bruno Aziza, Peter Horváth, Klaus Moeller, Frank Buytendijk, Ian Shore, James Creelman (special thanks for all your help with this book), Leif Edvinsson, Marc André Marr, Mark Graham Brown and Paul Niven. Of course, so many other individuals have also influenced my thinking and I hope all of them know who they are and how much I have valued any input and dialogue over the years.

My work at Cambridge University and Cranfield School of Management has also laid many foundations of my thinking. Especially useful were the insights from my doctorate training, which allowed me to develop my thinking on evidence-based management.

Of course I must thank the many people in companies and government agencies with which I have had the pleasure to work. Without all these organizations and their people it would have been impossible for me to develop my insights and tools. I therefore would like to thank all the many executives, managers and employees I have worked with over the years.

Finally, this book would never have been possible without the support from my wonderful wife Claire. Claire and our children Sophia and James are my clear priority in life and I am grateful that they support me. They provide so much inspiration to make the world a better place - and if this means helping organizations to become more intelligent and successful then this is a very small and humble, but hopefully worthwhile, step in the right direction.

CHAPTER 1

THE DATA-KNOWLEDGE CRUNCH

Evidence-based management is a simple idea. It just means finding the best evidence that you can, facing those facts, and acting on those facts.

Professor Robert I. Sutton, Stanford University

INTRODUCTION

To use a 19th century analogy to begin a book that explains how to overcome the 21st century challenges of converting ever-increasing amounts of data into insights that drive effective decision making might seem odd. But this is exactly what I will do, and for good reason.

Think of the California gold rush of 1849. People flocked en masse to that US state with the hope of making their fortunes from unearthing more of the gold that had just been found. A core tool, or technology, used by these prospectors was a gold pan, which would sift out gravel, sand, sediment and so on, but retain the heavier gold nuggets. By panning the endless tons of worthless silt, the prospectors hoped to find those few precious nuggets of gold that would make them rich men. A few did indeed become hugely wealthy, but most returned to their homes having either expended their investments without an adequate return (if any) or worse, bankrupt.

I now fast forward more than 150 years to the end of the first decade of the 21st century. Switching attention from gold prospectors to business managers - and analysing how the latter are attempting to secure financial and other gains from their particular economic and business landscapes - leads to a quite disturbing observation: most are behaving, and deploying interventions, in ways that are in reality little different to the gold prospectors of old.

Within enterprises today, business leaders are simply expected to pan masses of essentially worthless, or background, data, with the hope that somehow they will discover those golden nuggets that increase the wealth of the enterprise.

THE DATA AND INFORMATION EXPLOSION

To explain: thanks to a decade or so of breathtaking advancements in information and communications technologies humans now live in a world in which data, in all its forms, can be transmitted simultaneously to large numbers of people across the world by a single click of a button - and at the speed of light. Moreover, as a result of equally stunning technological improvements in data storage, much of the mass of data transmitted between employees resides somewhere in the enterprise - in databases, computer systems or other devices.

Within most medium to large enterprises the amount of data being exchanged and stored on a daily basis is almost incalculable: as is its value, because this data is the core material required for forming those ‘golden nuggets’ of insights that enable the enterprise leaders to make better decisions and so ultimately gain measurable and sustainable financial and other successes.

And we should not forget that having rapid access to the best information for decision-making purposes is not just about securing the obvious gains such as increased revenue, profit or market share. Today, it might be as much about ensuring survival in increasingly networked and connected globalized economies by getting early warning signals of potentially catastrophic market problems or other failings. For instance, the credit crunch of 2008 led to the collapse of long-established and venerable companies such as Lehman Brothers and many other household names would have been lost to history if it were not for large-scale financial interventions by national governments. The aftermath of the credit crunch has been the destabilizing of global markets and national economies. As much as anything these events shone a spotlight on what can happen when senior managers lack insight into what is happening in their organizations and markets and when bad decisions are being played out in fully networked, global, marketplaces.

Only the most foolishly optimistic would believe that in today’s globalized economies the cataclysmic economic events of 2008 are isolated events. The implications are clear. Decision makers require a quality and accuracy of decision-support that is quantum leaps superior to a few short years ago. The rewards from getting those few golden nuggets of knowledge into the right hands at the right time might be astronomical.

THE FAILURE TO TURN DATA INTO MISSION-CRITICAL INSIGHTS

Yet, and despite the mouthwatering benefits on offer (be that growth or survival), few organizations are deploying practical, reliable and replicable processes for unearthing those mission-critical insights. Research into the working practices of most organizations and their managers confirms that the ‘gold prospector’ analogy holds true. Many organizations are content to hoard data, in the mistaken belief that simply having the data available is in and of itself value-adding, leaving it up to individual decision makers to pan for those golden nuggets. Management writer David Apgar is correct when he says in his book Relevance that although new technologies, such as faster processors, bigger storage and optical fibres have made data storage easier, relevance has become less important: ‘Cheap information has tempted us to neglect relevance and led us into some bad habits,’ he writes (Apgar, 2008).

But not all organizations have fallen into bad habits. As one example, a global study (The Hackett Group, 2006) found that those finance organizations that were judged as world-class EPM (Enterprise Performance Management) performers generate significantly less reports than the non-world-class group - namely 691 reports per year per US$ billion in revenues compared to 1474, and the reports were also much shorter.

On first reading it appears that the world-class group was less productive and less value-adding than their non-world-class counterparts. However deeper analysis tells a very different story. The finance staff of the world-class group spent considerable time ensuring that the reports that they provided to business leaders focussed on the critical information that was required for decision-making purposes. They weren’t just throwing a mass of data at business leaders with the instruction to besieged managers that somewhere in the mass of pages delivered would be something of value. So, the unwritten message is that managers should get their knowledge prospecting pans out.

The world-class group recognized that they have a core responsibility to apply their analytic skills in the translation of raw data into knowledge. And the rewards to their firms of doing so were considerable. Over a three-year period the world-class group generated industry-relative equity returns that were more than twice that of the non-world-class group (The Hackett Group, 2006).

Research by the management consultancy Accenture confirmed the likelihood of greater stock market returns as a consequence of better analytics (Harris and Davenport, 2007). A survey of 371 companies found that 65% of top performers said that they had significant decision-support or real-time analytical capabilities - versus 23% of low performers. The same study found that 40% of top performers use analytics across the whole organization compared to 23% of low performers.

Yet, if organizations are today struggling to extract the greatest competitive benefits from their available data, there is no doubt that the scale of the challenge will grow significantly going forward, as year on year our capabilities to store and communicate data increase exponentially. To illustrate, as a result of the masses of data that it can access through myriad information and communication channels, the typical weekday edition of The New York Times contains more information than the average person was likely to come across in a lifetime in seventh-century England. However, research predicts that by the end of 2010 the world’s information base will be doubling in size every 11 hours (therefore, more than twice in each and every day!).

Humans are experiencing an almost unimaginable information explosion. We will soon reach the stage where it will be virtually impossible to point to a subject or topic for which there is a lack of data (although that does not mean that people will have articulated answers to the pressing questions related to those subjects/ topics, which is a central argument of this book).

What we are witnessing within most organizations, and across all sectors and industries, is that decision makers are being bombarded by an ever-expanding supply of data. This is placing them and their organizations under great strain, and led to what can be termed ‘the knowledge crunch’. This term suggests the organizational paralysis that is being experienced as a result of possessing large amounts of data, but being incapable of converting this data into the key information required to support effective decision making. It is safe to argue that although most organizations are drowning in data they are thirsting for the relevant information to support key decisions.

Put another way, the more data that is available the easier it is to miss the most crucial bits of information being sought. Most readers would, at some time, have completed a Google or similar search and spent a significant amount of time in an increasingly frustrating hunt through many and ultimately irrelevant links before finding that specific piece of information they were looking for. The fact is, the required information was there, it was just hidden.

INVESTMENT IN BUSINESS INTELLIGENCE

Of course, most business leaders are acutely aware that they might have somewhere in their organizations the data or information that they require and that it is simply hidden away in a system, database or some other storage facility. And most are equally and painfully cognizant of the fact that as they see their information stocks snowballing and as the amount of data that is gathered grows, there is an urgent need to be able to analyse that information in a way that can add value and bring competitive advantage.

Most leaders, however, would acknowledge that analytic skills are in short supply, as confirmed by research evidence. A recent global survey found that more than half of organizations (59%) believe that they do not have sufficient capabilities to analyse comprehensively their data, while as many as 87% feel that their analysis capabilities need to be improved (Marr, 2008).

As a matter of urgency, organizational leaders are asking themselves a question of strategic importance: how to retrieve and make strategic, competitive sense of the mass of data that they possess or could access. With an acknowledged lack of adequate business analytical capabilities, many organizations have turned to information technology (IT) solutions in the belief that this will answer the question and therefore solve their data and analytics problems. In 2007 alone, organizations spent more than $4 billion on licence revenue for so called business intelligence tools (software applications that allow people to analyse data; see Box 1.1 for a definition of business intelligence and other key terms).

Box 1.1: Defining some key terms

• Data comes in myriad forms, including numbers, words, sounds or pictures but without context (e.g., 15/3, 5, 68).

• Information is a collection of words, numbers, sounds or pictures that have meaning (e.g., on the 15th March at 5pm we were all at 68 Victoria Street).

• Knowledge is where we take in and understand information about a subject that then allows us to form judgements in order to make decisions and act on that knowledge. We do this by using rules about the world that we worked out through having lots of information from the past.

• Business Intelligence refers to technologies, applications and practices for the collection, integration, analysis and presentation of business information.

• Analytics refers to the use of data and evidence, statistical, quantitative and qualitative analysis, explanatory and predictive models, and fact-based management to drive decision making.

Indeed, the world’s leading IT research and advisory company Gartner (2008) reported that business intelligence was the number one technology priority for the third year in a row, and that it is seen as supporting the top three business priorities of improving business processes, attracting and retaining new customers. Clearly, business leaders firmly believe (or at least hope) that the implementation of business intelligence tools will resolve pressing business issues.

In spite of their massive investments into technological solutions, there is little doubt that organizations are still failing to convert data into strategically valuable knowledge. The fact is that software alone will not solve the decision-support crisis that organizations are facing, and neither should they expect it to do so. As I explain in this book, technology is simply an enabler of the data to knowledge metamorphoses.

To be fair, it is unsurprising that organizational leaders can be seduced into believing that IT can resolve many of the issues that give them sleepless nights. As far back as 1963 an article in Business Week predicted that, ‘the great day - when all the information for solving a management problem is only a push button away - is closer than you think.’ More than 45 years later we are still waiting for this ‘great day’ to arrive. Although the sensible among us realize that it never will, it is easy to see from the marketing literature that supports many of the business intelligence applications why many people might believe that analytics-related questions will indeed be answered by the simple pressing of a button.

You should also bear in mind that those at the top of organizations are experiencing IT as a ‘second language’ and have had to learn how it can be most effectively applied within organizations while the relevant technologies themselves were evolving at breakneck speed. When most of today’s senior managers entered the workforce, technology was little more than a mainframe computer that lived in a big room somewhere in the organization that only a few made use of or could understand. Personal computers might have been in the early stages of appearing on the desktop, but connective technologies such as the Internet and email were still unheard of outside of limited academic and military fields.

Those entering the workforce today, however, have a different story. They are experiencing technology as a ‘first language’, and are as fluent in the language of technology as they are in their spoken language. When these employees reach senior positions they will be well aware of the shortcomings of technology (or rather what it cannot do or should not be expected to do), irrespective of how advanced technological capabilities are at that time: and we can have little doubt that they will be extraordinarily advanced.

EVIDENCE-BASED MANAGEMENT

Rather than being deployed in isolation, to be fully effective IT and applications have to be used in close alignment with the business goals and the information and analysis needs of the people in the organization. Such alignment creates a dynamic in which it is much more likely that the right management decision can be made because technology can be used to facilitate the answering of focused performance questions and the solving of specific business problems. And the best management decisions tend to be those that are supported by relevant facts and insights. This is where Evidence-based Management (EbM), the focus of this book, comes in.

EbM is an approach used by a growing number of leading organizations (such as Google, Capital One, Harrah’s Entertainment and Tesco, to name but a few that I highlight within the book) to ensure that they collect the most relevant information to support the key decisions. These companies are doing much more than just collecting and storing data and information in large quantities. They are building their competitive strategies around data-driven insights. Babson College Professor Thomas Davenport, a leading thinker in and exponent of evidence-based approaches to management, explained that such organizations are becoming ‘analytics competitors’ (Davenport, 2006). He urged business leaders to

Use sophisticated data-collection technology and analysis to wring every last drop of value from all of your business processes. With analytics, you discern not only what your customers want but how much they are willing to pay and what keeps them loyal. You look beyond your compensation costs to calculate your workforce’s exact contribution to the bottom line. And you don’t just track existing inventories; you also predict and prevent further inventory problems.

Davenport (2006)

Davenport continued his ‘call to arms’, by exhorting organizations to

Make analytics part of your overarching competitive strategy, and push it down to decision makers at every level. You’ll arm your employees with the best evidence and quantitative tools for making the best decisions - big and small, every day.

Davenport (2006)

Although the EbM idea is intuitive and straightforward, most organizations seem to struggle with three aspects: (i) collecting relevant and reliable information; (ii) finding relevant data among the overwhelming amount of data available to them; and (iii) turning the data into information and knowledge that allows them to act on it.

This book provides a step-by-step guide for overcoming these challenges. This is achieved by deploying a five-step EbM framework, which is broadly described in Chapter 2 (and summarized in Figure 2.1 on page 14), with each step fleshed out fully in subsequent chapters. In addition to being of value to large firms, the framework is equally relevant to small- and medium-sized firms that might not have a massive budget for the latest business intelligence tools but which are also facing even greater challenges in relation to skills shortage and the need for better decision making.

But at the outset, it should be stressed that EbM is not some new and complex business theory that is difficult to put into operation or that requires substantial consulting support to get started and a well populated functional barony to maintain. Indeed Stanford University Professor Robert I. Sutton, another leading EbM proponent, rightly argues that ‘Evidence-based management is a simple idea. It just means finding the best evidence that you can, facing those facts, and acting on those facts’ (Sutton, 2009).

Evidence-based management in our private lives

When reflecting on Sutton’s statement, readers might be struck by the fact that in essence EbM is little more than the commonsensical approach to decision making that most of us apply intuitively on a regular basis in our private lives.

As a matter of course, everybody systematically collects, analyses and interprets as much data as possible, or as necessary, in order to be comfortable that they are making the right and most appropriate decisions. As just one of limitless examples, consider purchasing a house. Collecting the requisite evidence for this decision might include looking at crime statistics for the neighbourhood in which the house is located, speaking to potential future neighbours and looking at the published ratings of local schools, the availability of local transport links and networks, as well as spending time in the house to ensure that it meets all our living requirements. With all the required evidence collected, a yes/no decision can be made that is based on the best available evidence.

As another example, consider purchasing a car; collecting the evidence here might include reading expert reviews of the car that you are interested in, speaking to people who drive that model and the consulting of statistics on, as examples, fuel consumption, safety ratings and performance, as well as test driving the car to ensure that it meets your needs for comfort and ease of driving.

In short, therefore, without thinking of it as advanced management best practice, most of us make decisions on a day-to-day basis using sound EbM principles, which we would rightly argue as being ‘common sense’.

In essence, this book recommends taking effective decision-making approaches from our private lives and applying these within our working lives - oh, and underpinning this work-based decision-making process with powerful information technologies and applications to ensure that the evidence we collect is as robust, reliable and useful as possible. And once we have the insights, acting upon them.

CONCLUSIONS

Given the ever sharpening competitive and other pressures that organizations are facing in the early part of the 21st century, the time has arrived for a systematic, evidence-based approach to making decisions. The rewards for those organizations that inculcate such capabilities are as incalculable as might be the penalties for those that do not.

But succeeding with an evidence-based approach is not just about having business intelligence and analytics capabilities, although both are critical and indeed the core sub-components. Also required is an organizational culture that supports and values fact-based decision making, rather than decisions made purely on gut feel.

Importantly, a successful EbM implementation also requires an underpinning and far reaching mindset transformation from business leaders. Professor Sutton says that

[EbM is not about] doing what everyone else does, what you have always done, or what you thought was true. It isn’t an excuse for inaction. Leaders of organizations must have the courage to act on the best facts they have right now, and the humility to change what they do as better information is found. It isn’t a new idea and isn’t an original idea. Yet surprisingly few leaders and organizations actually do it - and those that do trump the competition.

Sutton (2009)

CHAPTER 2

THE EVIDENCE-BASED MANAGEMENT MODEL

Facts are stubborn things; and whatever may be our wishes, our inclinations, or the dictates of our passion, they cannot alter the state of facts and evidence.

John Adams (1735-1826), ‘Argument in Defense of the Soldiers in the Boston Massacre Trials’ , December 1770

INTRODUCTION

You have probably visited a doctor on many occasions. At these times you are sometimes fully aware of what the medical condition is (such as an infection) and the probable cure (such as antibiotics). There are other times, however, when you have a symptom, or range of symptoms, for which the underlying cause is uncertain, and this might trigger feelings of anxiety and deep concern. When such anxiety is present you rightly expect much more from your doctor than a cursory ‘once over’ and the writing of a prescription. You expect a thorough medical examination that might include a range of relevant tests to rule out critical illnesses (and so allay your worst fears) followed by an accurate diagnosis and an appropriate and effective course of treatment.

Figure 2.1: EbM model.

Put another way, you look to your doctor to collect all the relevant and available evidence relating to the symptoms in order to identify the underlying problem and its solution. You also expect the doctor to assemble this evidence from a range of sources, such as talking to you (the patient), consulting your medical records, taking blood and/or tissue samples and comparing the findings with documented case histories, as well as making use of his or her own experience and past observations.

From the perspective of this book the medical analogy is appropriate. As well as continuing the argument from the previous chapter that evidence-based decision making is something that we all apply in our everyday lives (from buying a house to getting a proper medical diagnosis) without questioning the logic, and that we assemble that evidence from a broad range of data and information points, the roots of EbM can be found in an established discipline from the medical world: evidence-based medicine.