Remember the Pepsi Challenge? It was a Pepsi marketing campaign at the height of the “cola wars” and suggested people liked Pepsi more than Coke. Malcolm Gladwell famously wrote that this was because people liked the sweeter taste of Pepsi when taking just one sip, but preferred the less-sweet taste of Coke if they were going to drink the whole can. True or not, I think it’s a useful metaphor for pop-ups1.
Remember pop-ups? You don’t have to! They are everywhere and frequently are the thing standing between you and the thing you want on a website. I bet you’ve already closed a few today.
There’s plenty of evidence that pop-ups “work.” When measuring effectiveness of pop-ups, that usually means looking at “conversions” for a link or form inside a message that interrupts a visitor.2
Conversions are good! Who doesn’t want more newsletter subscribers, more sales, more donations, or more volunteers? But pop-ups have a lot in common with the Pepsi challenge. The first “sip” is great, but then what?
An article I read about Groupon years ago has always stuck with me on this point. A long excerpt is warranted to understand how sweet the first sip can really be before it all turns sour (emphasis added):
There was one other part of the letter that says your biggest regrets are the moments that you let a lack of data override your intuition of what’s best for the customers. What did you mean by that? Groupon started out with these really tight principles about how the site was going to work, really being pro-customer. And as we expanded people in the company would say, “Hey, why don’t we try running two deals a day?” “Why don’t we start sending two emails a day?” And I’d think, That sounds awful. Who wants to get two emails every single day from a company. And they’d be like, Sure, it sounds awful to you. But we’re a data-driven company, so why don’t we let the data decide? Why don’t we do a test? And we’d do a test, and it would show that maybe people would unsubscribe at a slightly higher rate, but the increase in purchasing would more than make up for it. You’d get in a situation where it doesn’t feel right, but it does seem like a rational decision.
The problem was when you’re in hypergrowth like this, you don’t have time to see what is going to happen to the data in the long term. The churn would catch up with you. People would unsubscribe at higher rates, and then, before you know it, the service has just turned into something — if you look at Groupon now, it’s just this vestige of what it once was. There’s no real copywriting. It’s a marketplace of coupons. It’s still a service that a lot of people get a lot of value out of, but it doesn’t have the spirit it once did.
There are certain things you have to be religious about in the company. That’s what I’ve taken away from that: There are some things where you have to say, “I’m sorry. I’m not going to look at the data on that. This is just what we’re going to do. We know that it’s right, and there’s nothing that’s going to shake us from that.”
“The Super-Quick Rise and Even Faster Fall of Groupon
Former CEO Andrew Mason on what the roller-coaster ride felt like.”
There’s so much to unpack from Groupon’s experience:
- If you make a decision purely based on data—”Do pop-ups work?”—then you leave no room for your values, especially those about how you treat other people.
- When we say, “pop-ups work?”, what timescale are we considering? Will it always be true?
- It’s very easy to measure positive outcomes online like subscribes or donations. It is much much harder to measure the negative things like visitor emotional response or long-term attitude toward your brand.
Groupon still exists but is a shell of its former self. I’m not blaming their total collapse on sending too many emails, but I’m not not blaming it on that. Anyone who remembers Groupon remembers how the tide of public opinion and consumer sentiment turned on them very quickly. They started as a source of awesome deals and woke up one day mostly known for spamming people and being terrible for small businesses.
So back to pop-ups…
Even when they seem like a good solution on the surface, I’m admittedly hesitant to help an organization force that experience on a visitor.
I focus on working with most of my clients for the long-term, and that’s partially because I believe in the long-game when it comes to the web. Blogging over time builds solid SEO that withstands the whims of search engines. Thoughtful development of features and information makes visitors want to come back and give you money. And more respectful ways of asking for donations like banners at the top of the page or in emails probably work as well or better in the long-term.
Values matter. The way people feel about your organization matters. Long-term stability and dependability matter. Pop-ups can definitely taste sweet, but do you really want to drink the rest of the can?
- Or “soda-ups” if you’re not from the Midwest like me. 🥁 What!? How was I supposed to let that go? I had no choice. ↩︎
- Noteworthy detail: A significant percentage of pop-ups are not accessible or usable. That means some people may be unable to close or especially frustrated by a pop-up, regardless of whether they are interested in it. But I’m not even going to get into all the usability issues presented by many (most?) pop-ups this time. ↩︎