Recently, I gave a keynote presentation at the European Conversion Summit in Germany, and I revealed the best-kept secret of conversion optimization: Not All Tests Win.   Tweet this:

The best-kept secret of conversion optimization: Not All Tests Win
Tweet

Sadly, it’s true.

At WiderFunnel, we often share case studies of winning test results publicly. Most often, they show spectacular winning results and marketing insights.

While many people are understandably reluctant to acknowledge that they don’t always produce winners, I’m happy to. I’d like to tell you why.

Conversion Optimization Tests Don’t Always Win

I admit it. There are times that I’ve allowed conversion optimization test results to ruin my day.

You see, at WiderFunnel, we monitor every test on a daily basis so that we can make tactical changes to maximize ROI.

When there are tests on the dashboard that don’t show strong winning results, it’s hard not to let them get me down. I want to win big every time!

But We Do Win Most of the Time

Fortunately, the awesome team at WiderFunnel has an enviable success rate for tests.

I’ve had many conversations with internal testing teams and other agencies. Most companies report that around a third of their conversion optimization tests, out of every three, produce winners.   Tweet this:

For most companies, only 1/3 of CRO tests produce winners
Tweet An agency that runs tests regularly may get up to 50-60% wins, which is very good – about double the internal teams rate. With WiderFunnel’s system, we’re getting a much higher success rate. In our last quarter, 89% of tests planned by WiderFunnel produced a new champion page.   Tweet this:

89% of CRO tests planned by WiderFunnel produced a winner. Awesome!
Tweet

Awe-some!

I should be thrilled. But… there’s still that 11 percent that didn’t deliver a new winner.

Take a look at these two test results as examples. Staring at these every morning, day after day… well, it gets old, fast.

Clearly a test with the big winner is more fun to see. But, an inconclusive test doesn’t annoy me quite as much as it used to.   Tweet this:

Why inconclusive tests don't annoy me as much as they used to
Tweet

Why I Love Inconclusive and Losing Test Results   Tweet this:

Why I Love Inconclusive and Losing Conversion Optimization Test Results
Tweet

Over the years, I’ve learned and re-learned that those losing and inconclusive tests hold valuable insights.   Tweet this:

Losing and inconclusive conversion optimization tests hold valuable insights
Tweet

In fact, they can be so valuable that they should be called winners too.   Tweet this:

Why losing and inconclusive conversion optimization tests should be called winners
Tweet

We’ve seen over and over again that a commitment to continuous testing gets the best results. We often see the best results on the 2nd, 3rd and 4th tests on a page or template.

But, results don’t just come from repetition. Great results come from making insightful learning from each and every test. That’s important to understand: great results come from powerful insights.   Tweet this:

Great conversion optimization results come from powerful insights
Tweet

Turn Inconclusive Tests into Winners

Where that inconclusive test I mentioned earlier would have ruined my day in years past, now our team looks at it as an opportunity.

We analyze the results of inconclusive tests to understand the differences and produce hypotheses that will get better results.

In the case of this client with an inconclusive test, WiderFunnel’s next test round produced a statistically significant 16.1% winning result.

Sometimes the follow-up tests can be even more dramatically awesome. In the case of this client, who prefers to remain unnamed, the next round after an inconclusive result produced a triple-digit conversion rate lift.   Tweet this:

The follow up from inconclusive test can give triple-digit win
Tweet

Find Gold in the Losers

Like the ugly duckling that grows into a swan that lays golden eggs, you can analyze losers to gain insights. In other words, “There’s gold in them thar hills!“   Tweet this:

There's gold in them thar hills!
Tweet