I remember a dashboard from my last company. Our customer satisfaction score was a beautiful, climbing line. A solid 9 out of 10. We printed it out. We put it in a board deck. We felt great.
And at the same time, a quiet, nagging number was going the wrong way: churn. Not a crash, just a steady leak. The satisfaction score said they loved us. The bank account said they were leaving.
This is the survey trap. It’s the comfortable illusion that a number is the same as an answer. And it’s a trap I’ve walked into more than once, because the alternative is so much harder.
The Allure of the Spreadsheet
#Let’s be honest. Surveys are popular for a reason. They scale. You can send one to ten thousand people as easily as you can send it to ten. They produce clean, quantifiable data that you can put into a chart. It feels rigorous. It feels like science.
And for certain jobs, surveys are the right tool. They’re good for:
- Trending known metrics. If you need to watch your Net Promoter Score or satisfaction rating over time, a survey is your instrument. It’s the dashboard gauge.
- Validating a clear hypothesis. You think enterprise users care more about security features than startups do. A survey can confirm or deny that at scale.
- Segmenting a known audience. You want to know which features are most used by which customer plans. A survey can slice that data for you.
The key word in all of this is known. Surveys are excellent for measuring things you already know to look for. They are monitoring tools. They tell you the what, the where, and the how many. But they almost never tell you why.
And the why is where all the strategy lives.
The Uncomfortable Conversation
#Interviews are the opposite. They are messy. They are slow. They don’t scale. You can’t graph a conversation. The output is a pile of sentences, not a spreadsheet.
This is why we avoid them. But it is also precisely why they are so valuable. An interview is not for measuring. It’s for discovering.
When that satisfaction score was high and churn was creeping up, a survey would have been useless. What would we even ask? “Why are you churning even though you said you were satisfied?” The answers would be rationalizations, post-hoc justifications.
The real answer comes from an interview. It comes from a conversation that starts with a simple prompt: “Hey, I see you’ve been a customer for a year. Could you just walk me through what you were using our product for last week?”
In that conversation, you don’t get a number. You get a story. You hear the sigh. You hear them say, “Well, we used to use it for X, but then our process changed, and now it’s just easier to do it in a spreadsheet.”
Bingo.
That’s the friction the survey never thought to ask about. That’s the clue. An interview surfaces the questions you didn’t realize you should be asking. It gives you the raw material for your next good hypothesis. It also gives you a gift: the exact words your customers use to describe their problems. That language is gold for your marketing, your sales calls, and your product itself.
How to Ask Better Questions
#The value of an interview depends entirely on the quality of the questions. And the best questions are almost never direct. The goal is to get past the polite, rationalizing part of the brain and get to the real, lived experience.
It’s a shift from asking about opinions to asking for stories.
Don’t ask: “Why did you sign up for our product?” Ask: “Walk me back to the day before you signed up. What was happening that sent you looking for a tool like ours?”
Don’t ask: “What new features would you like to see?” Ask: “Tell me about the last time you tried to accomplish X with our product. Where did you get stuck?”
Don’t ask: “Would you pay for this feature?” (The answer is always yes). Ask: “What are you doing today to solve this problem? Are you paying for anything? How much time does it take?”
These questions get you past hypotheticals and into behavior. People are not very good at predicting their own future actions. But they are excellent historians of their own recent past. Your job is to be the journalist of that past, not a pollster of their future intentions.
A Rhythm for Learning
#This isn’t a case of interviews being good and surveys being bad. It’s about using the right tool for the job. The most effective founders I know don’t choose one. They use both, in a deliberate rhythm.
It looks like this:
- Explore with interviews. When you’re entering a new market, trying to understand a problem, or seeing a metric move in a strange way, start here. Talk to five to ten people. The goal is to find patterns and form a strong hypothesis.
- Validate with a survey. Once you have a hypothesis from your interviews (“We think people are churning because their internal processes are changing”), you can build a survey to test it at scale. Now your survey isn’t a wild guess; it’s a targeted diagnostic.
- Monitor with a survey. After you’ve used the insights to make a change, a survey can help you monitor the key metric you were trying to move. It’s back to being a dashboard gauge.
When that gauge starts to flash red again, you don’t panic. You don’t just send another survey. You go back to step one. You pick up the phone. It’s a cycle of learning that keeps you from reacting frantically to every wiggle in the data. It makes the whole process smoother.
The numbers tell you that something is happening. The stories tell you what it is. You need both. A dashboard full of numbers without the stories behind them is just a decoration.
So the next time a number on your screen surprises you, good or bad, resist the urge to build another chart to explain it.
Instead, find the person behind the data point. The real answer you need is probably in a sentence, not another cell.