“Don’t fall in love with the data.” – Frank Sesno
Not many companies ignore data these days since it’s often thought to be the secret sauce that’s going to lead to greater success.
Yet data can be an inexact science. Whether it’s erroneously predicting the winner of a presidential election or the number of expected flu cases in a certain year, data science is a technology that “can see things as never before, but also can be a blunt instrument, missing context and nuance,” finds The New York Times.
That’s why companies cannot be lulled into complacency when it comes to data, and must instead be ready to question it thoroughly, says Frank Sesno, a former CNN anchor.
Sesno, author of “Ask More: The Power of Questions to Open Doors, Uncover Solutions, and Spark Change,” says that teams and leaders can’t ignore their “gut instinct”, relying only on data to make decisions or predictions.
Experts say that data science is just another tool, and it’s designed to provide probabilities, not absolute answers. In addition, companies must understand that those who build the predictive models may have flawed assumptions or be mistaken about what data is most important to a company’s objective or strategy.
Data also can lead to teams not relying on their own knowledge and experience to come up with the best solutions. Researchers found in a study that 60% of radiologists asked to analyze a routine chest x-ray failed to detect that a collarbone was missing – because they were so familiar with data that trained them to expect to see one.
So, how do organizations use data to its best advantage? Experts say it begins with committing to a strategy that uses data – but not to the exclusion of anything else.
Asking the right questions
Sesno, who has interviewed five presidents and other world leaders, is now director of the School of Media and Public Affairs at The George Washington University. He says that his journalism training has taught him the power of asking the right questions, and he’s learned even more from people like Colin Powell, former Secretary of State and chairman of the Joint Chiefs of Staff, and Dr. Anthony Fauci, who was critical in cracking the HIV/AIDS mystery.
While data certainly adds to the overall picture when it comes to forming a strategy or developing goals, Sesno says that teams can’t “park their common sense” when using it and must still:
- Ask diagnostic questions. “What’s wrong?” “How do we know?” and “What are we not seeing?” are all ways to identify a problem. For example, if your company has introduced a product that has bombed, everyone might assume it’s just a product not desired by the public. But by using a focus group, you discover that people do indeed like the product, they just don’t know about it. While sales data show it’s not selling, the reality is that marketing problems are the real issue.
- Challenge the experts. “Data experts are intimidating because they are so smart. But don’t be so intimidated by them that you don’t ask simple questions such as ‘How do you know?’” Sesno says. “Questioning an expert can be daunting and difficult, but necessary.” Some questions to ask experts include: “Have you seen this before?” and “What else aren’t you telling me?”
- Learn from Colin Powell. “Someone like Colin Powell drills down when asking questions and tries to spin free of the data with his questioning,” Sesno says. “He seeks to define his mission and wants to know about alternatives.” When Iraq invaded Kuwait in 1990, Powell asked strategic questions about goals, resources, consequences, rationale and risk. Powell says he posed eight strategic questions, and only when he got eight “yeses” did he believe the president could move forward with a full-scale invasion of Kuwait. Of course, Powell is also known for asserting that U.S. intelligence was certain there were weapons of mass destruction in Iraq, a key reason for invasion by the U.S. More than a decade later, Sesno says Powell is still angry that a key source was not interrogated more thoroughly. “The ‘friggin director of the CIA should have asked! He should have asked his people, ‘What do we really know about this? Where did this come from? Is it multiple sourced?’” Powell says.
Taking a logical approach
Dr. Anthony Fauci is an immunologist and director of the National Institute of Allergy and Infectious Diseases. In the last year, he’s often been seen on national television discussing the Zika virus, but is also known for his breakthrough work on HIV/AIDS.
Sesno says that Fauci’s scientific method of confronting a problem, dealing with data and trying to find a solution provides insight on how others can assess whether the information they’re getting makes sense.
You can do that by:
- Determining the facts. What have you seen or know with a high degree of certainty? For example, when Fauci was looking at the AIDS epidemic, he knew that the Centers for Disease Control had found it mostly among young gay men who were dying of a kind of pneumonia that only hits those with compromised immune systems.
- Confronting the questions head on. Fauci’s team wanted to know what was going on, and why. Why were these young men dying of a disease that wasn’t supposed to attack healthy people?
- Developing and testing a hypothesis. Fauci hypothesized that there was a new autoimmune disease that was killing the young men. But only through experiments, tests, measures and documentation did Fauci see if his hypothesis would be viable. You have to be willing to think completely differently, as Fauci did, and be patient in finding answers. For example, you may be trying to decide why more customer complaints are coming in at the end of the month, and hypothesize it’s because the stores are doing inventory at that time so the busy staff isn’t responding as quickly to customers. Keep your hypothesis simple as you try to prove – and disprove – it.
- Seeing if the data will hold up. What do you need to measure? The number of customers? Customer complaints? Employees on duty during certain hours? Start collecting the data and then see if you can replicate it. Then do it again and see if your findings hold up. Are they supporting or contradicting your hypothesis? What data is weak? Inconsistent?
- Looking for proof. Does the data answer the question with which you started? Ask others what you may have missed. Do they see problems? Does your conclusion hold up?
Helping data scientists
Other experts say that questioning needs to be a two-way street. For example, those from various business units may not completely understand the data – but data scientists may also not fully understand the business. If data gatherers aren’t fully up to speed on what the business goals are, they may not even collect the right data in the first place or understand what critical pieces are missing.
"At some point, you need to get some measurable results that have some sort of impact. That's the way you know you have the right data and are leveraging the data," says Fabio Luzzi, Viacom’s vice president of advanced analytics and data science at Viacom.
In a Quora thread about asking better data science questions, several commenters suggested that data scientists need to spend more time understanding the subject before asking questions. Here are some of their suggestions:
- “Most importantly, always start with WHY. Secondly, a good trick is to flip a question around: instead of ‘Why are there so many accidents on this stretch of highway’ try to answer: ‘Why are there not more accidents on this stretch of highway?” -- Ricardo Saporta, who built the data science team at The Orchard
- “Better questions are asked if you have domain knowledge about the subject. For example, if the project involves home loan data, an understanding of the U.S. real estate market, mortgage rate and default trends as well as general consumer sentiment will definitely help you create interesting questions.” -- Ty Shaikh, Teaching Assistant at K2 Data Science
- “Nothing beats going on site and observing what is happening on the ground. If I need to predict machine failure, I always ask to visit the shop floor. I once followed a maintenance crew around for the afternoon as they punched their job data into their hardened mobile devices while wearing greasy gloves. And that was more insightful than weeks of research, meetings, and discussions about algorithms. Good questions flow easily once you know the context of your data.” -- Jason T. Widjaja, analytics and data science lead
In business today, coming up with better decisions is critical – but that can only happen by asking better questions, experts say. Without that, opportunities can be missed.
As American financier and presidential advisor once said: “Millions saw the apple fall, but Newton was the one who asked why.”