How to Know When You’ve Done Too Much Conversion Optimization

Can you do too much conversion optimization? When should you stop?

Recently, an attendee in a webinar I was presenting asked an interesting question:

“Is there such a thing as too much testing and/or optimizing?”   Tweet this:

Is there such a thing as too much conversion testing?

It’s not an unusual question. In many of my conference presentations, someone asks me something similar and I’d like to answer it today in our ongoing series of conversion optimization FAQs.

There are two sides to this question.

The Law of Diminishing Returns

Conversion rate optimization diminishing returns

That’s not how the web works, though

On one hand, the Law of Diminishing Returns suggests that your testing efforts will, at some point, produce lower value than your early testing. The early wins should give the biggest improvement as you harvest the “low-hanging fruit” by fixing the glaring problems. After that, large improvement opportunities should become more difficult to find.

Also, conversion rates can never reach over 100%, so that’s a hard stop.

If that’s true, then there must be a point where the cost of planning and producing tests outweighs the benefit of conversion rate improvement.

But, that’s not the way business works.

On the Other Hand, Should You Ever Stop Improving?

I used to think we’d see more impact from the law of diminishing returns, but we haven’t yet.   Tweet this:

I used to think we'd see more impact from the law of diminishing returns, but we haven't yet

The question, “When am I finished with conversion optimization?” implies that there is an “optimized” point – that there’s an ideal page. In practice, you’ll never hit the 100% barrier.

There is always room for improvement. There are always competitors innovating. There are always questions to answer.

Conversion rate optimization can never stop

What happens when you stop testing

If you stop testing, the rest of the web pulls ahead.   Tweet this:

If you stop testing, the rest of the web pulls ahead
That’s why we need Evolutionary Site Redesign.

I believe we are still in the early days of testing new web experiences.   Tweet this:

We are still in the early days of testing new web experiences
Innovative designers and UX architects are coming up with new experiences all the time. Some are terrible ideas and others are going to work better than our current standards. The beauty of the scientific approach is that you can kill the losers much more quickly and definitively without having to worry about them turning into sacred cows.   Tweet this:

The beauty of the scientific approach- you can kill the losers more quickly
The pace of business is too fast today to allow them to develop.

The question we started with also implies that the goal of conversion optimization is only to find that perfect conversion funnel.

But, that’s not the only goal of conversion optimization.

I Hate the Term “Conversion Rate Optimization”

I want to step back for a moment and challenge the entire conversion rate optimization concept.   Tweet this:

I want to challenge the entire conversion rate optimization concept

Part of the problem is in the name. “Conversion rate optimization” as a term is a misnomer.   Tweet this:

Conversion rate optimization as a term is a misnomer
It puts the emphasis on the measurement of success rather than the outcome of success.

The conversion rate metric should be the method to answer your important business questions, not the end in itself. Thinking about it in a broader decision-making context re-orients your ideas from just tweaking and tuning minor page improvements to be a way of making important decisions.

What we’re talking about isn’t just “conversion rate optimization.” In my conversion optimization book, I introduce the term “Strategic Marketing Optimization,” which I think is closer to reality, but it still implies perfection as a goal.

When you think about the discipline of iterative testing, its true value comes from delivering a better process for decision-making. It should be a continuous improvement process. I think it should really be called “Scientific Marketing.”   Tweet this:

Conversion rate optimization should be called Scientific Marketing

Your gut feeling is only indigestion

Comic by Timo Elliott

The question to ask yourself, then, is really:

“Is there such a thing as too much decision-making?”

or

“Is there such a thing as too much business improvement?”

The answer is clear.

You should test when you have a question to answer; when you need to improve your business; when your competitors are making improvements.

There is no “too much testing” just as there is no “too much deciding.”

We won’t get away from using the term Conversion Rate Optimization in the near future, and I’ve made a certain peace with it. But it should be viewed as more; it should be your crucible for decision-making.   Tweet this:

Conversion rate optimization should be your crucible for decision-making

7 Comments

  1. Converian May 27, 2013 at 8:16 am - Reply

    Fully agree that we are in early days of testing new web experiences. Many ecommerce owners have not passed the initial steps on understanding their traffic and jump on AB tests. From our experience +80% of AB testings are done in a wrong way and without a scientific methodology in place. I believe Conversion Rate industry it's at the beggining of its peak and we will see many innovations in this area.

  2. Brendan Regan May 31, 2013 at 8:29 am - Reply

    Nice post, Chris. The visuals help with these concepts, which aren't easy to digest! The other part of the original question might be phrased as: “Is there such a thing as too much testing and/or optimizing OVER TIME?” To your point, there is really no diminishing point for doing Optimization over time. There might be, however, a diminishing point if a site/organization tries to run too many Optimization activities at the SAME TIME. Either the test results could become polluted, or the organization might implode ;-)

    • @chrisgoward June 3, 2013 at 7:47 am - Reply

      Good point, Brendan! Running tests at the same time with the same visitors pollutes tests and can muddy results. Try to avoid implosion wherever possible :-)

  3. Shawn May 31, 2013 at 10:34 am - Reply

    Chris the important thing is whatever you will do its optimization or building your brand you have to watch it seriously , but not too much strictly. the images you have used in this article is help me lot to understand conversion optimization

  4. carl June 5, 2013 at 4:52 am - Reply

    Hm. Where you said that 'If you stop testing, competition will pass you..'

    Maybe for Target, Wallmart et al. But for the vast majority of the web, this is not the case. The vast majority of site owners have never heard of CRO, let alone have a comprehensive process of continuous improvement in place.

    but i agree with you on CRO being a silly wanky term :)

    • aaron84 July 30, 2013 at 7:28 am - Reply

      Yes, but it's not the vast majority of "unsophisticated" advertisers in the ad market that you should be concerned with. It's the other top performers. It's like how at the start of an NBA season, the Miami Heat don't care about 90% of teams and don't closely monitor what they're doing. Also, it should be noted that even if someone doesn't know what CRO is, and he is generally targeting a demographic like yours, he can still way overbid at scale, which would not directly help you. Thanks for posting…I enjoyed all the thoughts here.

  5. lizlambos September 9, 2013 at 6:09 am - Reply

    Definitely agree on "losing the forest looking for the trees" when you focus on only the funnel and not the actual core product and its relevance. Another case for continuing conversion rate optimization as a process is that in any evolving market best-practices, your user base or your product offering changes which makes continuous monitoring and improvement a must. I've found with a lot of sites even a slightly tweaked adword can re-shape the traffic and effect conversions for the previous best performing forms and calls to action on the site.

Leave A Comment

WP Socializer Aakash Web