Table of Contents
More Praise for Talent Intelligence
Title page
Copyright page
Dedication
Acknowledgments
About the Authors
1: Talent Measurement: Is It Measuring Up?
The Hidden Role of Talent Measurement
Why Talent Measurement Matters More Than Ever Before
The Ineffectiveness of Most Talent Measurement
Why Talent Measurement Is Not Working
The Purpose of This Book
2: Standard Measures of Talent: How Good Are They Really?
Finding Dependable Measures of Talent
Values, Social Skills, and Other Possible Signs of Talent
Finding a Way Forward
3: Rethinking Measurement: How Best to Predict Success
Three Principles for Deciding What to Measure
Measuring Potential
From Talent Measurement to Talent Matching
Changing Talent Measurement
4: Tools of the Trade: Eight Processes for Gathering Data
Two Basic Ways of Measuring Talent
Eight Less Biased Methods for Measuring Talent or Fit
5: Choosing the Best Methods and Tools for Your Business
Key Issues in Choosing Tools
Making the Choice
6: Building the Foundations for Talent Measurement
Why Foundations Are Often Missing
Four Things You Must Do to Make Measurement Work
Two Things You Need to Think About
The Final Foundation: Linking Measurement to Performance
Proceeding from a Sound Base
7: Getting the Most from Measurement Results
Using Results Consistently Well
Using Measurement Data to Do More
Building a Culture of Measurement
Spreading the Expertise
8: Sourcing the Expertise You Need
Sources of Expertise
Types of Experts
Using Vendors
On Track for Successful Implementation
9: Making Measurement Work
Four Steps to Making Measurement Work
Transforming the Market
Appendix: Frequently Asked Questions
Index
More Praise for Talent Intelligence
“Attracting, selecting, and retaining the right talent is a critical competitive edge for organizations today. In this sometimes shocking and eminently useful book, the authors lay out some of the challenges and complexities of talent measurement and show how by choosing the right combination of methods and tools—and implementing them consistently—businesses can produce good talent intelligence.”
—Dr. Nandani Lynton, leadership development director, A.P. Møller - Mærsk A/S
“Practical and provocative, this book provides a clear and comprehensive road map for cutting through the complexity and myths surrounding talent measurement. Essential reading for all HR professionals who want clear guidance on what to measure, how to measure it, and most importantly how to turn the data into genuine talent intelligence.”
—Ryan Eagar, Global Learning and Development, HEINEKEN International
“This book is revelatory—it challenges commonly held beliefs about talent measurement and clarifies what is and is not possible. It will change the way you think about what you do.”
—Alan Arnett, head of Talent Development, XL Group
“This book is essential reading for all HR professionals and managers who care about the management of talent in their organizations. It is a highly practical guide to talent measurement that makes this complex subject easy to understand and more possible to implement.”
—Martin Jahn, head of Global Fleet Sales, Volkswagen AG
“This intelligent book resets all assumptions and expectations about talent measurement and shows how businesses can better identify truly global potential in individuals in our search for an Asian way to corporate globalization.”
—Toyohiro Matsuda, head of Human Resources Development Asia, Mitsubishi Corporation
“In this highly accessible and useful guide to finding and evaluating talent, the authors have succeeded in simplifying the science of measuring talent for business readers. Although grounded in research, the book avoids much of the jargon typically found in research literature and focuses on what is most relevant to companies trying to identify talent. The authors summarize what works and what does not, highlight the problems associated with measurement tools, and offer practical and clear advice on how to approach this daunting task.”
—Nancy Tippins, senior VP, CEB Valtera; past president of the American Psychological Association's Organizational/Industrial Psychology Division
“This compelling book brings a scientific and systematic approach to a critical but previously underdeveloped element of talent management—namely, how organizations can make sure that the people they are hiring, retaining, and promoting are the best ones for their business. Arguing that companies cannot just leave this issue to chance or the natural skills of their managers, the authors show how firms can achieve a real competitive advantage in the talent market through a more sophisticated and effective approach to talent measurement. Informative, enlightening, and essential reading for business and human resources leaders.”
—Jackie Wong, executive director and CEO, Temasek Management Services
Cover image: Thinkstock
Cover design: Adrian Morgan
Copyright © 2013 by John Wiley & Sons, Inc. All rights reserved.
Published by Jossey-Bass
A Wiley Brand
One Montgomery Street, Suite 1200, San Francisco, CA 94104-4594—www.josseybass.com
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201-748-6008, or online at www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Readers should be aware that Internet Web sites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read.
Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax 317-572-4002.
Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.
Library of Congress Cataloging-in-Publication Data
Kinley, Nik, date.
Talent intelligence : what you need to know to identify and measure talent / Nik Kinley, Shlomo Ben-Hur. – First edition.
pages cm
Includes bibliographical references and index.
ISBN 978-1-118-53118-1 (cloth); ISBN 978-1-118-64008-1 (ebk); ISBN 978-1-118-64028-9 (ebk); 978-1-118-64007-4 (ebk)
1. Personnel management. 2. Employees–Rating of. 3. Employees–Recruiting. I. Ben-Hur, Shlomo, 1962– II. Title.
HF5549.K4994 2013
658.3'01–dc23
2013007356
To Connie, for her love and support.
For keeping the world turning while I wrote this.
And to Lukas, for his patience and for reminding me that
there are other things than writing books.
Yes, Papa can play now.—N.K.
* * *
To my beloved sisters Orly, Yula, and Inbar,
each talented in her own very special way.—S.B.H.
Acknowledgments
As with any other such undertaking, many people have contributed to this book. There are some without whom this book would never have happened. There are others who have helped us develop our thinking and form our ideas, both recently and over the years. And then there are those who have helped us write and hone the text. It is a long list, and we are grateful to them all.
Unfortunately, there is not enough space to name everyone, but there are some we absolutely must mention—people who have directly contributed time and effort to help us write this. At the top of this list has to be Lindsay McTeague, our brilliant copyeditor at IMD, and the fantastic Kathe Sweeney, Alina Poniewaz, Bev Miller, Carol Hartland, and Alan Venable at Jossey-Bass. Then, in alphabetical order, are Alan Arnett, Laurence Barrett, John Bassett, Georg Bauer, Stephanie Bird, Bill Byham, Jacqueline Davies, Ryan Eagar, Samantha Halverson, Peter Hartmann, Helen Hopper, Christopher Howarth, Martin Jahn, Fiona Laing, David Lawton, Andrew Lopianowski, Nandani Lynton, Rab MacIver, Mikkel Madvig, Farzana Maudarbaccus, Laura McKeaveney, Shauna McVeigh, Gabby Parry, Eyal Pavel, Elisabeth Pircher, David Ringwood, Peter Saville, Robert Sharrock, Nancy Tippins, Caroline Vanovermeire, and Jackie Wong.
About the Authors
Nik Kinley is a London-based independent consultant who has specialized in the fields of measurement and behavior change for over twenty years. He has worked with CEOs, factory-floor workers, life-sentence prisoners, government officials, and children. His prior roles include global head of assessment for the BP Group, head of learning for Barclays GRBF, and senior consultant with YSC, the leading European assessment and development consultancy.
He began his career in commercial roles, before spending the next decade working in forensic psychotherapy. Ten years ago, he returned to working with organizations and since then has worked with over half of the top twenty FTSE companies, identifying and developing talent across the globe. He now specializes in consulting with businesses to help them build their talent intelligence and enhance the performance of their people, and consulting with vendors to help them develop talent and learning-related products and services. He holds a master's degree in systemic psychotherapy and a bachelor's degree in psychology from the University of London.
* * *
Shlomo Ben-Hur is an organizational psychologist and a professor of leadership and organizational behavior at IMD business school in Lausanne, Switzerland. His areas of focus are the psychological and cultural aspects of leadership and the strategic and operational elements of talent management and corporate learning. In addition to teaching leadership on two of IMD's top programs for senior executives, he creates programs for and consults with a wide variety of organizations across the globe.
Prior to joining IMD, he spent twenty years in the corporate world, most recently as vice president of leadership development and learning for the BP Group based in London and earlier as chief learning officer of DaimlerChrysler Services AG in Berlin. He earned his doctoral degree in psychology from the Humboldt University of Berlin. He holds a master's degree in industrial/organizational psychology and a bachelor's degree in psychology and political science from Bar-Ilan University in Israel.
* * *
Contact the authors at inbox@measuringtalent.com or check out their blog with the latest news, information, and advice on talent intelligence at www.measuringtalent.com.
1
Talent Measurement
Is It Measuring Up?
The challenge with most companies' talent intelligence is that it is just not that intelligent.
Having good talent intelligence—an accurate understanding of the skills, expertise, and qualities of people—is essential for the people decisions that every business makes. If they are to avoid randomly hiring and promoting people, all companies need to evaluate and gauge individuals' talents. It is a basic and fundamental task, one that every manager and organization does, and one that everyone agrees needs to be done well. Yet all the available evidence suggests that it is not.
The reason lies in talent measurement—how organizations go about gathering and using information about the talents of their people—because it seems that this crucial task is often taken for granted, not well understood, or undertaken in ways that limit its value to firms.
This book is about why this is so, what has gone wrong, and what organizations can do to rectify the matter. And they do need to rectify it because the world is changing in ways that mean they will no longer be able to get away with not doing it well. To thrive in the coming decades, firms are going to need good talent intelligence, and they are going to need to use it in ways that deliver real value and competitive advantage for them. And to achieve this, they are going to have to get talent measurement right. The good news is that they can.
Talent measurement is the use of various methods and tools to gather and use information about individuals' talents. There is no one way of doing it. Some organizations rely on the intuition of their leaders and simple interviews; others employ sophisticated online tests. Both, though, have the same purpose: to identify whether job applicants and current employees have the abilities, expertise, and characteristics they need to help both them and their businesses thrive and be successful.
As a task, talent measurement is often hidden away, part of bigger and broader processes. Yet it is there. It is key to recruitment, promotion, high-flier identification, restructuring layoffs, organizational design, individual development, competence assurance in technical roles, and due diligence for mergers and acquisitions.
Unsung it may be, but talent measurement is a fundamental foundation of modern talent management. It is a basic building block in successfully managing workforces, helping identify who adds value right now, and who could do so going forward. And although it may have been a low-profile activity to date, it is about to have its day in the sun.
Talent management is changing, and as it does so, it is leading businesses increasingly to focus and rely on talent measurement.
Almost fifteen years ago, McKinsey declared that a “war for talent” was coming, and it seems they got it right.1 Globalization and shifting population demographics are causing competition for talent to rise steadily and persistently and making it harder than ever before for businesses to find the talent they need.
In the West, only 18 percent of firms say they have enough talent in place to meet future business needs, and more than half report that their business is already being held back by a lack of leadership talent.2 Worryingly, 75 percent of businesses report difficulty in filling vacancies too.3 The temporary increase in available workers created by the downturn is not helping either, as there is evidence that all the choice is making it more difficult to spot the best people.4
The situation is generally not as critical in emerging markets, but this will change. In China, for example, the predominantly manufacturing base of its economy has largely protected it from these concerns up to now. Yet as service industries and the use of knowledge workers grow and the impact of the country's one-child policy is felt, China too will face these challenges. The war for talent is going global.
It is not actual war, of course, but there will be casualties and there will be winners. We know that businesses that are better at talent management and better able to find and keep the best people tend to outperform their industry's average return to shareholders by around 22 percent.5 In fact, making good hiring and promotion decisions can have a bigger impact on market value than creating a customer-focused environment, improving benefits, or having good union relationships.6 And amid stronger competition for talent, these performance advantages for companies that are effective at identifying and managing talent are likely to increase.
Realizing this, alert organizations are turning to talent management for solutions and investing in it too A recent U.S. Department of Labor report predicted that over the next ten years, the number of people in human resources (HR) and talent management professions will grow at more than double the rate of the general workforce.7
Driven by all this attention and investment, talent management is changing. Perhaps most notable, and arguably long overdue, it is becoming far more data led. People data have become currency, and workforce analytics is the buzzword of the moment. The idea is simple and compelling: to manage talent and make good personnel decisions requires knowing what you need, what you have, and what is available. And to make this possible, new software systems have emerged that promise to help you gather, manage, and use talent information more effectively than ever before.
You might assume that talent measurement would be at the heart of this analytical talent management revolution, but, oddly, this has not typically been the case so far. Instead, these talent systems tend to use data such as demographics and distributions—that is, workforce composition. This type of administrative information does have uses, but it is limited in terms of what you can do with it and the value you can add with it. So while many of these new talent management tools are undeniably impressive, they are, like all other systems, only as good as the data you put into them. And in this respect, they are lacking.
A few larger companies have sought to rectify this by putting talent measurement at the heart of these systems. Google, predictably, is ahead of the curve when it comes to people data. Unsure of whether it was hiring the best applicants, the company started developing a comprehensive database that captured information about current employees' attitudes, behaviors, personality, biographical information, and job performance. This database has allowed Google to develop an algorithm for predicting which applicants are most likely to succeed at the company.8 It is too early to judge how effective the algorithm is, and this kind of approach would not be suitable for all businesses. Yet it is clearly more sophisticated in its approach than mere demographics and has the potential to yield far more value.
Other organizations are following suit. For example, a major UK retail bank recently linked the results of its employee engagement survey to administrative data on people, measurement data, and customer service feedback scores for individual bank branches. As a result, it was better able to understand what the business and branch managers needed to do to improve the customer experience.
So businesses are beginning to realize the potential of measuring talent systematically and combining talent data with other information to produce insights of real business value. This may sound like good news and a great opportunity. And it is. But it can be seized as an opportunity only if talent measurement works and produces good-quality intelligence, which is where things get worrying.
Unfortunately, the vast majority of organizations are ineffective in how they measure talent. Even among companies that are measuring talent effectively, most are using the information it provides in ways that mean they derive only a small margin of the real value it can deliver.
For example, surveys show that less than one-third of business leaders rate their company's selection processes as effective.9 Indeed, while they see selection as the most important task of talent management, they also view it as the least effective.10 This is not limited to just hiring either. The results for other talent identification processes—such as promoting, benchmarking, or identifying potential future leaders—are not much better. This may be hard to hear, and your first instinct may be to dismiss it or rationalize it away. But it gets worse.
Over the past thirty years, businesses have invested heavily in trying to find the best people, to the extent that this period has witnessed the development of a global talent identification industry. There is the corporate recruitment market—the headhunting firms whose collected annual revenues prior to the downturn were estimated to be in excess of US$10 billion worldwide. And then there is the specialist talent measurement market, estimated to be worth more than US$3 billion per annum globally.
With all this investment, you might expect to find that businesses had significantly improved their ability to identify talent and hire the right people. Yet when we compare research from thirty years ago into how well new employees do with research from today, this is not what we find. Instead, the rate of failure among new employees seems to have risen. Thirty years ago, it was estimated that about one-third of all new employees failed.11 Today, reported failure rates range from 30 to 67 percent, with an average of about 50 percent.12
What is shocking about this is not so much the high rate of failure or even the rise in failure rates over the past few decades, but that the measurement industry has had no discernible impact on these rates. Somehow, despite massive investment in measurement and the widespread adoption of sophisticated methods and tools, we do not appear to have achieved meaningful improvements in failure rates. There is no shortage of case studies showcasing individual organizations that have done great work in this area, but across the board, this success is just not shared.
In almost any other area of business, investing that kind of money and not making a dent in failure rates would be unacceptable—or at least it should be. As a number of commentators have noted, in a world where organizations are placing an unerring focus on results, they seem to tolerate surprisingly low success rates when it comes to hiring and promoting people.13 Indeed, it is hard to think of any other area of management where such poor performance would be tolerated.14
That is not to say that the task is easy. The sheer complexity and number of variables involved is often understated, and some of the reasons and circumstances that cause people to fail are not predictable.15 For this reason, we are unlikely ever to reach 90 percent of our people decisions being highly effective. But we should be doing better than we are.
So what is going on? One obvious possibility that springs to mind is that current talent measurement methods do not work or even that “talent” cannot be measured. But decades of research have unequivocally demonstrated that some measurement methods and tools are better at predicting both overall performance and individual elements of it than the traditional, basic selection procedure of using just unstructured interviews.16 In fact, study after study has driven this point home until it is no longer a matter of debate. Moreover, if accurate talent measurement were not possible, then no one would be making any progress. But that is not what we see. Instead, there appear to be pockets of excellence surrounded by a general lack of progress.
Studies show, for example, that effective talent measurement in recruitment and promotion processes can lead to reduced turnover, improved performance levels, and faster integration and time to full productivity. Indeed, effective talent measurement in hiring executives has been shown to result in companies being eight times more likely to hire someone they keep and go on to later promote.17 And it is not just success rates that good measurement can have an impact on. The use of some measurement tools has been shown to be able to cut absenteeism and decrease both accidents at work and employee theft.
So measurement can work and the growing use of it over the past thirty years should have had a greater impact. Somehow, somewhere, something has gone wrong. And it is a critical issue, because if businesses cannot make talent measurement work, the rest of their talent management activities are likely to come up short.
In our work with organizations around the world looking at the issues they face in talent measurement, we have found five common challenges:
Each challenge by itself can significantly limit the ability of measurement to have the sort of impact we would expect. But in our experience, most organizations are struggling on all five fronts.
It is difficult to do something well if you do not fully understand it, and talent measurement is a highly technical business. Indeed, it has its own subareas of expertise, such as the mathematics of test design, which many measurement professionals themselves do not fully understand. Not everyone needs to know all the technical details, of course, but even at an operational level, measurement can be complex.
For starters, you need to know what you want to measure. Companies usually know this at a broad level—for example, they want to know if someone is a potential leader for the future. Yet knowing what specifically to measure can be a lot harder. Is it behavioral competencies and, if so, which ones? Should you look for intelligence? Personality? Ambition? And how do you know which qualities make the biggest difference in which situations?
Moreover, what if the best test to use in order to predict future performance also happens to be the one that shows most bias against some racial groups? Or what if an organization wants to use one consistent measurement tool across all its offices around the world, but some countries have regulations controlling which measures can be used? These, in fact, are some of the most common complexities that businesses encounter, and they can create significant problems.
The complexity does not end once you have worked out what to measure. You then have to choose the right tool for the task, and here you encounter the thorny issue of how to ensure that you are accurately measuring what you set out to assess. For example, we encountered a major global bank that in its Singapore office used a respected test to assess the intelligence of all job applicants. Yet it was assessing all of the candidates, no matter what their background, using tests written in English. The logic was simple enough: it wanted to be able to benchmark candidates' intelligence with that of people in the UK head office. The complexity the firm did not grasp, however, was that it was not getting a true, accurate reading of intelligence because candidates' language abilities were affecting how well they did on the test. What the company should have done was to use intelligence tests in candidates' native language, and if it wanted to assess their English ability as well, then it should have also administered a separate language skills test.
Of course, complexity in itself is not a problem. It becomes problematic only when the complexity is not recognized or is underestimated. So it is unfortunate that many vendors, in an effort not to scare potential customers away, tend to downplay the complexities and keep the inner workings of measurement out of sight. It is commercially understandable and, from a customer's viewpoint possibly, preferable. After all, we live in a world in which convenience, keeping things simple, and “just-do-it” solutions are valued. But understanding complexity can sometimes be necessary for things to work effectively, and this is certainly true for measurement. Measurement is a complex issue, and if it is to be done well, it needs to be treated as such. Failure to do so will mean that whatever you do, the chances are it will not work.
Adding to the complexity is the fact that finding the right solution can be difficult. The measurement market is awash with a mass of different methods and tools, and the choice can be bewildering. Information about which tools should be used when and which work best tends to come from one of three sources:
In theory the best source of information should be academic research because it is the only reliably objective source. Yet surveys show that HR professionals and business leaders alike rarely read academic journals and often consider research contradictory or irrelevant.18 And who can blame them? The research literature can be hard to access and even harder to understand. As preparation for this book, we read over a thousand articles, so with some authority, we can confirm that they can be difficult to understand and downright mind numbing.
The result is that businesses tend to be relatively uninformed about measurement research and have to rely instead on what vendors tell them. Yet without objective sources of information, HR and business leaders often report feeling intimidated by the apparent expertise of vendors—or at least unable to question or challenge what vendors tell them.
The importance of this is that organizations need to question and challenge what they hear. Some excellent vendors, services, and tools are on the market, but there are estimated to be over two thousand test publishers in the United States alone, and only a minority of them engages in any proper validity studies.19 So only a small percentage of vendors can say with any objective authority that they know that their measurement methods genuinely work.
Moreover, even when they do have evidence of the quality of their tools, this information cannot be taken at face value. The reason lies in the worrying trend of reporting bias: the tendency for people to publish only positive results or ones that further their arguments or products. Measurement is, of course, a business, and we understand that in this commercial environment, vendors need to present themselves well. But recent research shows that reporting bias is far more prevalent than you might expect in an industry that professes to be grounded in science.
At a broad level, for example, there is evidence that academic research findings are less favorable about the success of measurement than research produced by vendors.20 More specifically, studies have identified reporting bias by some very well-known psychometric test publishers.21 The publisher of one of the most globally used personality tests, for instance, states that the tool has great validity, yet a review by a respected independent body has concluded that “the test suffers from questionable reliability and unknown validity. Its use is not recommended.”22
Probably the most public example of the issue is the tale of emotional intelligence. In the mid-1990s psychologist and author Daniel Goleman brought to the fore the idea that emotional skills are important for leadership success. On the back of the book came a number of tools claiming to measure emotional intelligence, and with them came claims that they could account for 80 percent of the factors that determine success.
Almost twenty years on, however, there is now overwhelming independent research showing that emotional intelligence measures are actually some of the less effective predictors of success. This does not mean that emotional intelligence is not important for leadership. It simply means that measures of it are nowhere near as good at predicting success as initially claimed. Yet if you Google these measures, you will find the same original weighty claims still being made by some big-name vendors selling them, without mention of the decades' worth of independent research findings to the contrary.23
This prevalence of biased validity figures makes the recent actions of one of the biggest test providers in the world all the more concerning. It appears to have changed its contractual terms to prevent independent research into the validity of its tests without its approval and permission. In our view, this throws any kind of pretense about objective science straight out of the window.
So not only is measurement a complex, technical, and all-too-often impenetrable field, but knowing who and what to trust is not easy. Little wonder that when recently asking for our help in setting up a new talent measurement process, one of the biggest companies in the world said that it felt “vulnerable” to the market.
It may feel at this point that there is no easy way to determine if measures and tools actually work. But all you need to know is which questions to ask and what to look out for in the answers. And businesses have the opportunity here not just to find out which tools work, but also to change how the measurement market works and make it easier to navigate. For example, if they stop using vendors who do not provide proper validity information, those vendors will either start producing it or disappear. And if firms simply refuse to use vendors that prohibit independent research into their tools, then these vendors will soon revert to allowing it. Far from being hopeless, the reality of the situation is that armed with just a little knowledge, you can make a big difference.
The choice of what measures and tools to use is complicated by the fact that they have traditionally been developed without considering how organizations use them. As a result, researchers and vendors have sometimes developed measurement methods that look great in theory and are strongly able to predict performance, but have not been used or are not much liked by businesses.
The biggest example of this can be seen in the academic articles expressing surprise that businesses so frequently ignore one of the most accepted findings in measurement research: that structured interviews tend to be far better able to predict performance than unstructured interviews. This surprise betrays a lack of understanding that the purpose of interviews for businesses is not just to predict performance. Interviews also need to leave candidates with a positive impression of the company and give managers a chance to gauge what their working relationship with candidates might be like. Yet proposals to heavily structure interviews, which in the strictest sense does not allow for any unscripted questions, clearly do not acknowledge these additional objectives. Some researchers have suggested that the reason structured interviews are not used more is that their benefits have not been clearly communicated.24 The reality is that they simply do not meet business needs.
Furthermore, researchers and test developers for the most part have taken the objective of measurement to be predicting job performance. At present, the yardstick for whether a measure is viewed as valid or effective is if it can predict who receives the best overall performance ratings. This certainly sounds reasonable, and indeed it is, in that this kind of information can be important in making people decisions. Yet the emphasis on predicting performance has been so strong that it has come at the expense of also trying to develop tools that can predict other factors that may affect a person's success.
For example, hiring managers are usually not only interested in who is the most able or could theoretically perform best. They also tend to be interested in factors such as whether potential new employees will get along with them, fit with the company's values, or work well with their coworkers. These issues may not sound as immediately compelling as candidates' likely overall level of performance. Nevertheless, they are critical to individuals' longer-term success and are some of the most frequent reasons people eventually “fail” in a role, despite considerable apparent ability. Let's face it: if your manager does not like you, then chances are that you are not going to succeed no matter how good you are on paper.
To be fair, there has been a shift in recent years. Vendors are beginning to produce more user-friendly tools and are starting to look at a broader range of factors that lead to success. And, of course, some vendors are better at this than others. But in general, the move has been late, slow, and minimal, and it has some way to go.
Knowing what to measure and how to measure it may be the most obvious challenges facing businesses when it comes to gauging talent, but they are not the biggest ones. In fact, in spite of everything we have said about how hard it is to know what works, the choice of measurement processes is usually the easiest thing to get right. It is everything else that is much harder for businesses to do effectively—things such as how they use measurement outputs to make decisions, how well they integrate measurement activities with other processes, and the degree to which they use measurement data to inform their broader people strategy.
The importance of these implementation issues is that if insufficient attention is paid to them, they can fundamentally limit the value and usefulness of the intelligence that talent measurement produces. Yet insufficient attention is exactly what these issues typically receive.
Working both within organizations and as external consultants to them, we have lost count of the number of times we have been asked to help set up a new measurement process or identify the best tool to use. But rarely have we been asked about how to make more use of measurement data or how to develop a company's attitudes and approach to using the data. Yet in our experience, it is such seemingly peripheral issues that all too often constrain and limit the potential impact and value of measurement processes.
For example, one of the easiest wins with measurement data collected in recruitment processes is to use this information to help tailor initial developmental support for new joiners. Yet research shows that only 19 percent of firms do this.25 It is certainly common enough to hear talk of how important such issues are, but the reality is that all too often, they are an afterthought and so not implemented effectively, if at all.
The shame in all of this is that measurement can do so much more than merely guide and support individual people decisions and development. Indeed, not doing more with the results is probably the single biggest missed opportunity that exists with measurement. With the advent of talent analytics, the situation is changing as businesses look more closely at what they can use measurement data for, but they have a lot of catching up to do.
Finally, to meet the first four challenges successfully, you either need to have measurement expertise yourself or access to someone who does. Unfortunately, the people who make decisions about measurement issues and manage vendors frequently have little such expertise themselves and little independent expertise available to them.26 As a result, they often either use the wrong measurement processes for their needs or use them in ways that limit their impact.
An increasing number of companies do employ experts to help them navigate the market and manage processes. Yet these roles are typically at a fairly junior level and predominantly tactical in nature. As a result, they can have little influence on measurement strategy. Of course, many smaller firms cannot afford or justify employing specialists and so have to find and rely on external experts and vendors. In itself, there is nothing wrong with this. But with little knowledge of the field, it is not easy to be effective in choosing and managing vendors.
In an effort to navigate the field, firms often resort to what Peter Saville, one of the fathers of modern measurement, calls faith validity. This is the tendency to become attached to particular tests or vendors that are familiar instead of objectively considering what works best.27 Yet surveys show a large diversity of opinion among HR professionals on which measurement methods work best.28 And wide gaps appear to exist between what research tells us is the best approach and what practitioners and businesses believe and in fact do.29
For example, the highly popular Myers-Briggs Type Indicator (MBTI) continues to be used in selection processes, even though the test distributors repeatedly assert that it should not be applied in this context.30 As a more general example, there is the continued use of graphology in places like France despite a mass of evidence demonstrating its lack of efficacy.31 And we recently came across a business that claimed that the measure it used to help identify which candidates to hire was right 95 percent of the time. Yet this belief in the tool appears to be unfounded. The company has never evaluated it, and in the tool's technical manual, the vendor suggests that it could account for only 3 percent of the reasons that people succeed. Something does not add up.
This is not to say that businesses need to know everything about measurement—far from it. Of course, the more access to expertise they have, the better. But we are convinced that even smaller firms without access to independent expertise can successfully manage measurement and make it work for them. All they need is a basic understanding of the challenges, knowledge of what fundamental questions they should always ask, and an awareness of how they can make the most of measurement.