Description
I can’t speak for all authors, but I feel that a book—especially one based largely on ongoing research—is never really finished. This is precisely what editions are for. In the time since the publication of the second edition of this book, I continue to come across fascinating published research about the power and oddities of human decision making. And as my small firm continues to apply the methods in this book to real-world problems, I have even more examples I can use to illustrate the concepts. Feedback from readers and my experience explaining these concepts to many audiences have also helped me refine the message.
Of course, if the demand for the book wasn’t still strong six years after the first edition was published, Wiley and I wouldn’t be quite as incentivized to publish another edition. We also found this book, written explicitly for business managers, was catching on in universities. Professors from all over the world were contacting me to say they were using this book in a course they were teaching. In some cases it was the primary text—even though How to Measure Anything (HTMA) was never written as a textbook. Now that we see this growing area of interest, Wiley and I decided we should also create an accompanying workbook and instructor materials with this edition. Instructor materials are available at www.wiley.com.
In the time since I wrote the first edition of HTMA, I’ve written a second edition (2010) and two other titles—The Failure of Risk Management: Why It’s Broken and How to Fix It and Pulse: The New Science of Harnessing Internet Buzz to Track Threats and Opportunities. I wrote these books to expand on ideas I mention in earlier editions of How to Measure Anything and I also combine some of the key points I make in these books into this new edition.
For example, I started writing The Failure of Risk Management because I felt that the topic of risk, on which I could spend only one chapter and a few other references in this book, merited much more space. I argued that a lot of the most popular methods used in risk assessments and risk management don’t stand up to the bright light of scientific scrutiny. And I wasn’t just talking about the financial industry. I started writing the book well before the financial crisis started. I wanted to make it just as relevant to another Hurricane Katrina, tsunami, or 9/11 as to a financial crisis. My third book, Pulse, deals with what I believe to be one of the most powerful new measurement instruments of the twenty-first century. It describes how the Internet and, in particular, social media can be used as a vast data source for measuring all sorts of macroscopic trends. I’ve also written several more articles, and the combined research from them, my other books, and comments from readers on the book’s website to create new material to add to this edition.
This edition also adds more philosophy about different approaches to probabilities, including what are known as the “Bayesian” versus “frequentist” interpretations of probability. These issues may not always seem relevant to a practical “how-to” business book, but I believe it is important as a foundation for better understanding of measurement methods in general. For readers not interested in these issues, I’ve relegated some of the discussion to a series of “Purely Philosophical Interludes” found between some chapters, which the reader is free to study as their interests lead them. For readers who choose to delve into the Purely Philosophical Interludes, they will discover that I argue strongly for what is known as the subjective Bayesian approach to probability. While not as explicit until this edition, the philosophical position I argue for was always underlying everything I’ve written about measurement. Some readers who have dug in their heels on the other side of the issue may take exception to some of my characterizations, but I believe I make the case that, for the purposes of decision analysis, Bayesian methods are the most appropriate. And I still discuss non-Bayesian methods both because they are useful by themselves and because they are so widely used that lacking some literacy in these methods would limit the reader’s understanding of the larger issue of measurement.
In total, each of these new topics adds a significant amount of content to this edition. Having said that, the basic message of HTMA is still the same as it has been in the earlier two editions. I wrote this book to correct a costly myth that permeates many organizations today: that certain things can’t be measured. This widely held belief is a significant drain on the economy, public welfare, the environment, and even national security. “Intangibles” such as the value of quality, employee morale, or even the economic impact of cleaner water are frequently part of some critical business or government policy decision. Often an important decision requires better knowledge of the alleged intangible, but when an executive believes something to be immeasurable, attempts to measure it will not even be considered.
As a result, decisions are less informed than they could be. The chance of error increases. Resources are misallocated, good ideas are rejected, and bad ideas are accepted. Money is wasted. In some cases, life and health are put in jeopardy. The belief that some things—even very important things—might be impossible to measure is sand in the gears of the entire economy and the welfare of the population.
All important decision makers could benefit from learning that anything they really need to know is measurable. However, in a democracy and a free-enterprise economy, voters and consumers count among these “important decision makers.” Chances are that your decisions in some part of your life or your professional responsibilities would be improved by better measurement. And it’s virtually certain that your life has already been affected—negatively—by the lack of measurement in someone else’s decisions in business or government.
I’ve made a career out of measuring the sorts of things many thought were immeasurable. I first started to notice the need for better measurement in 1988, shortly after I started working for Coopers & Lybrand as a brand-new MBA in the management consulting practice. I was surprised at how often clients dismissed a critical quantity—something that would affect a major new investment or policy decision—as completely beyond measurement. Statistics and quantitative methods courses were still fresh in my mind. In some cases, when someone called something “immeasurable,” I would remember a specific example where it was actually measured. I began to suspect any claim of immeasurability as possibly premature, and I would do research to confirm or refute the claim. Time after time, I kept finding that the allegedly immeasurable thing was already measured by an academic or perhaps professionals in another industry.
At the same time, I was noticing that books about quantitative methods didn’t focus on making the case that everything is measurable. They also did not focus on making the material accessible to the people who really needed it. They start with the assumption that the reader already believes something to be measurable, and it is just a matter of executing the appropriate algorithm. And these books tended to assume that the reader’s objective was a level of rigor that would suffice for publication in a scientific journal—not merely a decrease in uncertainty about some critical decision with a method a non-statistician could understand.
In 1995, after years of these observations, I decided that a market existed for better measurements for managers. I pulled together methods from several fields to create a solution. The wide variety of measurement-related projects I had since 1995 allowed me to fine-tune this method. Not only was every alleged immeasurable turning out not to be so, the most intractable “intangibles” were often being measured by surprisingly simple methods. It was time to challenge the persistent belief that important quantities were beyond measurement.
In the course of writing this book, I felt as if I were exposing a big secret and that once the secret was out, perhaps a lot of apparently intractable problems would be solved. I even imagined it would be a small “scientific revolution” of sorts for managers—a distant cousin of the methods of “scientific management” introduced a century ago by Frederick Taylor. This material should be even more relevant than Taylor’s methods turned out to be for twenty-first-century managers. Whereas scientific management originally focused on optimizing labor processes, we now need to optimize measurements for management decisions. Formal methods for measuring those things management usually ignores have often barely reached the level of alchemy. We need to move from alchemy to the equivalent of chemistry and physics.
The publisher and I considered several titles. All the titles considered started with “How to Measure Anything” but weren’t always followed by “Finding the Value of ‘Intangibles’ in Business.” I could have used the title of a seminar I give called “How to Measure Anything, But Only What You Need To.” Since the methods in this book include computing the economic value of measurement (so that we know where to spend our measurement efforts), it seemed particularly appropriate. We also considered “How to Measure Anything: Valuing Intangibles in Business, Government, and Technology” since there are so many technology and government examples in this book alongside the general business examples. But the title chosen, How to Measure Anything: Finding the Value of “Intangibles” in Business, seemed to grab the right audience and convey the point of the book without necessarily excluding much of what the book is about.
As Chapter 1 explains further, the book is organized into four parts. The chapters and sections should be read in order because each part tends to rely on instructions from the earlier parts. Part One makes the case that everything is measurable and offers some examples that should inspire readers to attempt measurements even when it seems impossible. It contains the basic philosophy of the entire book, so, if you don’t read anything else, read this section. In particular, the specific definition of measurement discussed in this section is critical to correctly understand the rest of the book.
In Chapter 1, I suggest a challenge for readers, and I will reinforce that challenge by mentioning it here. Write down one or more measurement challenges you have in home life or work, then read this book with the specific objective of finding a way to measure them. If those measurements influence a decision of any significance, then the cost of the book and the time to study it will be paid back many-fold.
CONTENTS
- Preface to the Third Edition
- Acknowledgments
- About the Author
- PART I: The Measurement Solution Exists
- CHAPTER 1: The Challenge of Intangibles
- The Alleged Intangibles
- Yes, I Mean Anything
- The Proposal: It’s about Decisions
- A “Power Tools” Approach to Measurement
- A Guide to the Rest of the Book
- CHAPTER 2: An Intuitive Measurement Habit: Eratosthenes, Enrico, and Emily
- How an Ancient Greek Measured the Size of Earth
- Estimating: Be Like Fermi
- Experiments: Not Just for Adults
- Notes on What to Learn from Eratosthenes, Enrico, and Emily
- Notes
- CHAPTER 3: The Illusion of Intangibles: Why Immeasurables Aren’t
- The Concept of Measurement
- The Object of Measurement
- The Methods of Measurement
- Economic Objections to Measurement
- The Broader Objection to the Usefulness of “Statistics”
- Ethical Objections to Measurement
- Reversing Old Assumptions
- Notes
- Note
- CHAPTER 1: The Challenge of Intangibles
- PART II: Before You Measure
- CHAPTER 4: Clarifying the Measurement Problem
- Toward a Universal Approach to Measurement
- The Unexpected Challenge of Defining a Decision
- If You Understand It, You Can Model It
- Getting the Language Right: What “Uncertainty” and “Risk” Really Mean
- An Example of a Clarified Decision
- Notes
- Notes
- CHAPTER 5: Calibrated Estimates: How Much Do You Know Now?
- Calibration Exercise
- Calibration Trick: Bet Money (or Even Just Pretend To)
- Further Improvements on Calibration
- Conceptual Obstacles to Calibration
- The Effects of Calibration Training
- Notes
- Notes
- CHAPTER 6: Quantifying Risk through Modeling
- How Not to Quantify Risk
- Real Risk Analysis: The Monte Carlo
- An Example of the Monte Carlo Method and Risk
- Tools and Other Resources for Monte Carlo Simulations
- The Risk Paradox and the Need for Better Risk Analysis
- Notes
- CHAPTER 7: Quantifying the Value of Information
- The Chance of Being Wrong and the Cost of Being Wrong: Expected Opportunity Loss
- The Value of Information for Ranges
- Beyond Yes/No: Decisions on a Continuum
- The Imperfect World: The Value of Partial Uncertainty Reduction
- The Epiphany Equation: How the Value of Information Changes Everything
- Summarizing Uncertainty, Risk, and Information Value: The Pre-Measurements
- Notes
- CHAPTER 4: Clarifying the Measurement Problem
- PART III: Measurement Methods
- CHAPTER 8: The Transition: From What to Measure to How to Measure
- Tools of Observation: Introduction to the Instrument of Measurement
- Decomposition
- Secondary Research: Assuming You Weren’t the First to Measure It
- The Basic Methods of Observation: If One Doesn’t Work, Try the Next
- Measure Just Enough
- Consider the Error
- Choose and Design the Instrument
- Note
- CHAPTER 9: Sampling Reality: How Observing Some Things Tells Us about All Things
- Building an Intuition for Random Sampling: The Jelly Bean Example
- A Little about Little Samples: A Beer Brewer’s Approach
- Are Small Samples Really “Statistically Significant”?
- When Outliers Matter Most
- The Easiest Sample Statistic Ever
- A Biased Sample of Sampling Methods
- Notes
- Notes
- CHAPTER 10: Bayes: Adding to What You Know Now
- The Basics and Bayes
- Using Your Natural Bayesian Instinct
- Heterogeneous Benchmarking: A “Brand Damage” Application
- Bayesian Inversion for Ranges: An Overview
- The Lessons of Bayes
- Notes
- CHAPTER 8: The Transition: From What to Measure to How to Measure
- PART IV: Beyond the Basics
- CHAPTER 11: Preference and Attitudes: The Softer Side of Measurement
- Observing Opinions, Values, and the Pursuit of Happiness
- A Willingness to Pay: Measuring Value via Trade-Offs
- Putting It All on the Line: Quantifying Risk Tolerance
- Quantifying Subjective Trade-Offs: Dealing with Multiple Conflicting Preferences
- Keeping the Big Picture in Mind: Profit Maximization versus Purely Subjective Trade-Offs
- Notes
- CHAPTER 12: The Ultimate Measurement Instrument: Human Judges
- Homo Absurdus: The Weird Reasons behind Our Decisions
- Getting Organized: A Performance Evaluation Example
- Surprisingly Simple Linear Models
- How to Standardize Any Evaluation: Rasch Models
- Removing Human Inconsistency: The Lens Model
- Panacea or Placebo?: Questionable Methods of Measurement
- Comparing the Methods
- Example: A Scientist Measures the Performance of a Decision Model
- Notes
- CHAPTER 13 : New Measurement Instruments for Management
- The Twenty-First-Century Tracker: Keeping Tabs with Technology
- Prediction Markets: A Dynamic Aggregation of Opinions
- Notes
- CHAPTER 14: A Universal Measurement Method: Applied Information Economics
- Bringing the Pieces Together
- Case: The Value of the System That Monitors Your Drinking Water
- Case: Forecasting Fuel for the Marine Corps
- Case: Measuring the Value of ACORD Standards
- Ideas for Getting Started: A Few Final Examples
- Summarizing the Philosophy
- Notes
- APPENDIX: Calibration Tests (and Their Answers)
- Index
- CHAPTER 11: Preference and Attitudes: The Softer Side of Measurement