In a deluge of data look for surprises, not confirmation
Interesting article shared by Google Re-work on avoiding confirmation biases
JENNIFER GARVEY BERGER AND WENDY BITTNER, CULTIVATING LEADERSHIP MAY 22, 2017
Have you ever tried to find a clear pathway but ended up drowning in data that seem to make everything more murky instead of less? Before you dive into data collection, try a bit of question reflection to make sure you understand what sorts of data will, and won’t, be useful.
“Juicilicious” was entering a new market (in this case, China), and trying to understand the particular tastes of these Chinese consumers. The team started collecting data — focus groups, industry trends, market surveys. They shipped trial Juicilicious products to China and collected feedback from consumers. They studied their competitors. They analyzed taste and ingredients. Before long, they found themselves so mired in data that they could hardly store it all, much less analyze it, with no clear insights. They figured they simply hadn’t collected the right data, so they set off to collect more... and so the cycle continued.
To break the cycle, you have to resist some of your old patterns and ask a couple of new questions that will help you take a complexity friendly mindset.
1. Is the situation you’re interested in predictable or unpredictable?
This first question might be one you’ve never asked before, but it could make all the difference. If you’ve done it once, is it reasonably likely that you will apply prior methods and get the same outcome? Assembly lines, supply chains, hospital admissions procedures — all of these should be predictable. In predictable challenges, experience, quantitative data, and probability-based analysis are your friends. If it’s worth the effort to wade through the data, do the analysis, sift out the noise, and find the answer.
But if the situation is unpredictable and heavily context dependent, you’ll need a different way to tackle the challenge — and your expertise and old analyses might be your enemy, so you’ll have to find a way to gently put them aside. Organizations are filled with complex challenges: culture change, acquisitions, entering a new market — basically anything that involves people changing their behavior. Just because you’ve been successful once doesn’t mean the same approach will be successful next time because even slightly different conditions lead to different outcomes. You need to think and act differently with the deluge of information.
2. Am I trying to prove what I already believe?
Once you see that you’re dealing with something unpredictable and complex, you’ve got to check your assumptions. In complexity, you want to use analysis to seek out what you’re missing — not to confirm what you already suspect to be right. If you’re trying to justify why your initiative is the best option or to convince your boss that her idea is wrong, you’re not actually learning from the data, but cooking it — a dangerous approach in a complex world. With all the data that swirl around us, it’s easy to use data to “prove” just about anything.
Instead, you’re looking for creativity — new ways of approaching a situation that might create new kinds of value. Rather than using data to confirm, you want to use your data to create surprise, to broaden options, to move you from proving what you believe is probable to exploring what might be possible.
3. What’s surprising or unexplainable in the data?
These questions are likely to open up new avenues to the possible rather than keeping you trapped in the probable. And don’t just ask these questions of yourself. Ask them of your colleagues too, so that you can begin to expand your access to surprise through multiple perspectives — and increase your number of creative options.
Watch out that you don’t unconsciously search for “The Best” insight, or “The Best” surprise. We often “listen to win” as we use the ideas or insights of our colleagues to support or build the answer we want the most. In complexity, you want to “listen to learn” so that you expand your vision to include new possibilities.
Each point of surprise — if you’re intentionally looking for it — creates the seed of an experiment that you hadn’t considered before. Test different hypotheses and then act quickly to take advantage of what moves you in the desired direction.
When Juicilicious realized that they were trying to make clear something that was in fact never going to be clear, they decided to take a different approach. They looked for ways to use their data to act in uncertainty and fail faster. They decided to take one insight — a surprise — that emerged from the data each week, and use that to create one juice recipe they could test quickly. And rather than shipping these quick-trials to China to test there, they decided to test them in a “Juice truck” they would drive around the Chinatown areas in cities close to their production facilities. Asking these different questions helped them to give up searching for certainty about what to do next, opened them up to the surprise of possibility, and led to an entirely unexpected — and successful! — line of products released domestically to recent immigrants, a population they had never before considered serving.
Jennifer Garvey Berger and Wendy Bittner are partners at Cultivating Leadership, a coaching group that helps leaders and organizations not only tolerate complexity but learn to use it to their advantage to become more agile and creative. Jennifer has 17 years of developing leaders in organisations and coaches leaders and senior teams facing dramatic change in their workplace. Wendy was a Senior Manager of Leadership Development and Senior Engagement Manager at McKinsey & Company for eight years.