Survivor’s Bias: Why It’s A Problem And How To Avoid It

17 Nov, 2021 By Bobby Klinck
Survivor’s Bias: Why It's A Problem And How To Avoid It

As people in the online marketing space, we need to recognize and understand that the information we’re seeing is coming at us through a particular lens – that of survivor’s bias.

In case you’re not familiar with this term, survivor’s bias is a concept that’s not specific to the online marketing world or even the marketing world. The idea behind it is that you tend to hear about only the best-case scenario results.

For example, in the case of car seats, you may think it’s okay not to put children in car seats because you hear from many adults who say they weren’t placed in car seats as kids and yet they’re fine. The catch is you won’t hear from those who didn’t make it.

Survivor’s bias usually comes to us in two variants: success stories of those who embraced the “All-In Mantra” and examples of “The Testimonial Trap.” That means we need to develop some savvy so that we recognize survivor’s bias when we encounter it and not make rash decisions.

The “All-In” Mantra

I started thinking about Survivor’s Bias as I was listening to the audiobook version of a book that’s taking the online marketing world by storm. It’s Alex Hormozi’s $100 Million Offers.

Alex Hormozi is an example of what I think of as The All-In Mantra of survivor’s bias. Entrepreneurs, not just online business owners, generally tend to lionize the people who went all-in and took ridiculous risks.

Hormozi talked about a time when he was at the cusp of a launch and suddenly had no money available. The only thing he had was a credit card with a $100,000 credit limit, and he decided to basically “go all-in” and put all his expenses on this credit card. He was in a situation where he was allowing tons of debt to pile up. You can see how this creates a story of being willing to bet on yourself.

To be candid, what he did was a horrible idea. If you have $100,000 on a credit card and you have no money to your name, maybe you shouldn’t go $100,000 into debt to start a business. It worked for Hormozi, which is why we’re hearing about it, but it won’t work for most people.

Another “All-In Mantra” story is that of the late Tony Hsieh of Zappos. In his book Delivering Happiness, he tells this story about how he and a friend created a business in college that they then ultimately sold where he walked away with about $30 million. He took that money and invested it in Zappos and kept investing more and more to the point that literally, he was having to liquidate and sell all his assets. He had gone “all-in” on Zappos but within months, Zappos was a failure.

People hear stories like these and then think they should do as Hormozi and Hsieh did: go all-in. I hear these stories and it makes me queasy because I recognize that the big problem has to do with Survivor’s Bias. What do I mean by that?

The Problem With All-In

Well, who are the people who are writing books after they’ve gone all-in? It’s the people for whom taking the risk worked out well. 

The people who went all-in and flopped and never built a successful business don’t write a book. Maybe they do, but nobody reads it.

And so, what we end up hearing about are the stupid risks that people took and we think, “Maybe that’s what I should do too because it’s gonna work out as long as I’m willing to invest in myself.”

I’ve been guilty of this myself. I’ve been telling my origin story of how in 2017, I spent a ton of money and went into the red in my online business. The difference is I’ve always said, “Don’t be like me. What I did was dumb.” I actively counsel people not to take stupid risks.

Examples of risky things entrepreneurs contemplate doing because of these successful “All-In” stories are investing in expensive coaching programs and other things. I’ll never forget the story of someone who talked about how they invested in a $40,000-plus coaching program upfront before they were making money. I hear that and all I can say is, if you’re a millionaire and you’ve got $40,000, that’s cool. You can afford it. But in my opinion, it’s still probably not a good investment if it’s a coaching program.

Don’t Build From A Place Of Fear

People will say that it’s good to put your back against the wall because that will get you invested. But I don’t believe that to be true. People can be invested, no matter what. If you’re only invested because you’re scared shitless, that’s not a good place to be coming from. Building a business from fear is not a good place to be.

To me, building a business means you have to make the right decisions. You must be in a place where, even if you want to succeed, you can choose actions that even if unsuccessful won’t land you in the poorhouse. When you go into a launch with an energy where you’re basically saying, “I have to succeed, or I’m going to be broke”… those launches don’t work.

You need to be careful about hearing and listening to people who are talking about going all-in. Going into debt, investing, mortgaging their futures… you only tend to hear from the people for whom those risks worked out well.

For every one of those people who succeeded, there are many who didn’t. We don’t hear about them because they didn’t survive in the entrepreneurship space.

Don’t Give Into The Pressure

Don’t believe everything that people say about a product. For instance, take it with a grain of salt when you hear someone say, “I invested in a $30,000 master class when I didn’t have the money and that was the best decision I ever made.”

If you don’t have the money, don’t do it. That means if you’re having to figure out creative ways to eat in order to buy a coaching program, don’t buy that coaching program. And if you’re struggling to pay your rent or a mortgage, don’t buy an online course.

Building a business is hard, and getting to the point of making it work is not just a three-month process. For most people, it’s a multi-year process. You have to accept that and you have to be willing to put in that work.  

This isn’t to say, however, that you can never take on debt. Different people have different perspectives on debt and when you should go into debt and when you shouldn’t. I believe it takes discipline to use debt the right way.

So again, the “All-In Mantra” where you spend all your money, drain your 401k, go into debt, that is problematic and toxic. We need to stop lionizing the stories of people who went all-in and succeeded with it because we’re never hearing from the people who did it and failed.

The Testimonial Trap

The other place where survivor’s bias comes up is in what I’m calling “The Testimonial Trap.” This has relevance in our niche because the way that many people are presenting testimonials in the online marketing space, particularly those that are doing results-based testimonials, is illegal under the Federal Trade Commission (FTC) law.  

Under FTC laws, if you’re marketing, you’re not allowed to cherry-pick your best cases. The FTC has said very clearly that you aren’t allowed to simply report the results of your most successful customers. You’re required to tell people what the typical results are, and no one is doing that.

As a consumer, you must recognize the “Testimonial Trap” when you’re hearing testimonials from people in situations, for example, where you’re thinking about buying an online program.

People who don’t have success with a program largely don’t ever report their results to the person who created the program. So, the people who are running programs, for the most part, only get feedback and testimonials from the few people who are experiencing success.

It’s a problem when you’re only hearing from a small subset of people who are the most successful with your product or program. There’s self-selection happening in the results that people are getting, and we’re only hearing from the people at the top.

Testimonial Realities

Most people only get testimonials from only a small percentage of people, maybe from only 1 percent to 5 percent, who have used the product or been in a program. 

So, the question becomes, what is the average success rate in online programs and how do I assess if it’s survivor’s bias or something else? 

I’m simply going to tell you that this is survivor’s bias because I know plenty of people who’ve taken plenty of online programs and who’ve done the work, yet they haven’t had success. Those people tend to outnumber the people who’ve taken it and had success.

A Different Approach To Testimonials

Instead of focusing on results when using testimonials, you can talk about features, like how a program was simple, easy to use, and helped people take action. Be a savvy consumer and recognize that when you see a Facebook ad where someone is offering a program on creating a sold-out coaching program in 30 days and presenting it as though creating a coaching program is going to be easy, you won’t fall into the Testimonial Trap.

Recognize that we’re hearing promises that are most likely based upon only the top results. If they collect 100 testimonials, they’ll tend to cherry-pick the 20 that are the best.

Testimonials are a classic example of survivor’s bias. We’re only hearing from the people who survived, made it work, and have been successful. We’re simply hearing the best-case scenario.

Is it possible that you could match the best-case scenario? Well, of course, anything’s possible. But just like it’s possible that if I buy a lottery ticket I could win a jackpot of hundreds of millions of dollars, it’s probably not going to happen, but it’s possible. 

Develop Your Savvy

There are so many things out there being offered to us. Instead of mindlessly buying, remember that you’re being marketed to, and you need to be savvy.

You need to recognize that almost everything you hear is from the lens of survivor’s bias, whether it’s a case of the “All-In” Mantra or The Testimonial Trap.

When you hear any kind of claim, ask yourself if you’re only hearing from the people who succeeded and made it work. There are probably way more people who tried the exact thing, and it didn’t work for them. If you want to have an honest perspective on what’s likely to happen when you follow a success secret or when you purchase a product or program, you must take Survivor’s Bias into account.

Want to know more about how to successfully build an online business? Be sure to join BADA$$ Online Marketing University (BOMU). It’s my FREE training program with courses on online business foundations, legal foundations, email marketing, and more, designed to help entrepreneurs like you.

Leave a Reply

Your email address will not be published. Required fields are marked *

Bobby Klinck

Harvard Lawyer and Online Entrepreneur