Don't do 100+ customer discovery interviews on your idea

I met with a first-time startup founder yesterday who had an interesting idea for a new medical device.

After some time spent discussing the concept and reviewing her lean canvas I asked about what she planned to do next. Her response?

“Do 100 customer discovery interviews to see if there is a need for this.”

She meant well, but was given bad advice somewhere along the way…

Why that’s bad advice

I won’t “name and shame,” but some startup accelerators, incubators and entrepreneurship programs at universities preach the idea of doing 100+ discovery interviews before launching a new product.

Sure enough, she admitted she had gone through an entrepreneurship program at her alma mater and that's where she got this advice from.

It’s well-intentioned advice. Doing any discovery research is better than nothing. Surely doing 100+ means you’re going to deeply understand the customer and their problem(s), right?

It doesn't necessarily hurt per se. The problem is three fold…

First, it’s intimidating

Doing 100 discovery interviews can take hundreds of hours to do. You have to find people to talk to, schedule a time to speak with them live, interview them and then analyze what they had to say. That can take anywhere from 1-3 hours per interview. Now multiply that by 100 and you’ll see the issue…

If you’re working nights and weekends on your startup idea (like most founders are when they're starting out) it could take you months to get that many interviews done. That's a massive barrier for many founders!

So what do some founders do instead? Skip the critical discovery stage and just start building a MVP. This is especially common if they’re not going through a cohort-based program that is keeping them accountable.

I suspect some startup programs set the goal of 100+ interviews to identify the true hustlers willing to put in the work. Fair enough. Some might also do it to see if a founder can even find 100 people to talk to in their target market. Also fair.

Regardless of the "hidden" reasons it presents a significant and unnecessary barrier.

Second, it’s not “one size fits all”

The founder I was talking with could develop a prototype on her own - without significant upfront investment. It would take her a few months to take the product to market. Despite being a medical/health-related product it won't be subject to regulatory hurdles. The buyer is clear, and it isn’t an enterprise sale with long sales cycles and multiple decision makers.

While she still needs to do some discovery work, her risks are relatively low. Doing 100+ discovery interviews would be overkill (if it wasn’t a bad idea for other reasons). If the potential investment to build the product was in the millions and it could take years to develop, then maybe I could justify doing that many interviews upfront.

However, there is one more reason it's overkill...

Third, it’s repetitive

You’ll start to hear the same themes repeated over and over again after a certain number of interviews. The term for this is reaching "data saturation," and it refers to the point at which you've reached diminishing returns in collecting more data.

There is a lot of debate and various factors that influence when data saturation is reached, but a safe bet is anything after 30 interviews. I've experienced this many, many times running discovery research studies in my career. Go ahead and try this for yourself and you'll see what I mean.

Anything past 30 interviews within a given audience and you're wasting your time.*

*Note: Sample sizes for traditional quant surveys are a different topic and don't follow these same recommendations. Discovery is fundamentally qualitative though...

So how many discovery interviews should you do?

If you're doing early discovery work on a new concept I’d recommend no fewer than 20 and no more than 30 customer discovery interviews. Anything less than 20 and you may not surface a significant enough range of opinions/experiences. Anything over 30 and you'll hear the same themes repeated over and over.

But there's a catch...

My recommendation applies under two scenarios:

1. This has to be with a single target audience. If you have multiple target customers then you should do 20-30 discovery interviews per customer segment.
2. You need to deliberately change up the order of questions between interviews (I'll share why in a future post).

For those past the initial discovery and validation stage

If you’re past the early problem/solution validation phase you’ll want to shoot for a steady pace of 2-5 interviews per week. If you can do more, great. But that’s a good starting point to build a general understanding of your customers. More on doing continuous customer discovery to come in future posts...