How to Prioritize Conversion Rate Optimization Tests Using PIE

One of the most common questions I’m asked about conversion optimization is:

“How do I figure out where to test first?”

It’s a smart question to ask. You can’t test all pages at once. With limited time and resources to commit and, most importantly, limited traffic to allocate to each test, test prioritization is an important part of your conversion rate optimization plan.   Tweet this:

Test prioritization is an important part of your conversion optimization plan

The alternative happens when companies start with the areas they believe are obvious. You might consider the popularity of the page or look for under-performing pages, or start with pages your HiPPO hates.

Those are all things to consider, but they don’t give you the whole picture.

Optimizing the optimization process is often just as important as the tests themselves. Prioritizing where you invest energy will give you better returns by emphasizing pages that are more important to the business.

The Three Criteria to Prioritize Pages

You need to consider three criteria to prioritize which pages to test and in which order: Potential, Importance and Ease.   Tweet this:

Consider 3 criteria to prioritize which pages to test: Potential, Importance and Ease

Potential

How much improvement can be made on the pages? Although I’ve yet to find a page without some potential for improvement, you can’t test everywhere at once and you should prioritize your worst performers. This should take into account your web analytics data, customer data and expert heuristic analysis of user scenarios.

Importance

How valuable is the traffic to the pages? Your most important pages are the ones with the highest volume and the costliest traffic. You may have identified pages that perform terribly, but if they don’t have significant volume of costly traffic, they aren’t testing priorities.

Ease

How complicated will the test be to implement on the page or template? The final consideration is the degree of difficulty a test will take to get running on a page, which includes technical implementation and organizational or political barriers. The less time and resources you need to invest for the same return, the better. This includes both technical and “political” ease. A page that would be technically easy may have many stakeholders or vested interests that can cause barriers. I’m looking at you, home page.

You can quantify each of your potential opportunities based on these criteria to create your test priority list.

We use the PIE Framework table to turn all of the data inputs into an objective number ranking.   Tweet this:

Create a PIE table to create an objective ranking of test opportunities

conversion rate optimization what to test first

Your Prioritization is Unique

There are no standard rules for which pages are best to prioritize. Your website lives in a unique target market, including factors like your competition, seasonality, and internal cultural environment; all of these affect how your site is used and should be optimized. The priority rating you give each of your potential test pages will depend on this unique business environment.   Tweet this:

The priority of your test pages depends on your unique environment

By using the PIE Framework, you’ll remove gut feeling from the decision and focus your team on an objective, relative ranking.

Applying the PIE Framework

Recently, on my morning walk to work, I recorded a story of how we applied PIE with a conversion optimization client.

How do you prioritize your conversion optimization test opportunities?

Add your comment below!

12 Comments

  1. Rob Kingston May 20, 2013 at 5:48 am - Reply

    I normally use a MoSCoW rating alongside Time/Cost and expected uplift.

    I used to score tests based on Ease, Benefit and Cost but the resulting score never worked out all too well. PIE sounds very tasty though.

  2. Michael Straker May 21, 2013 at 4:41 pm - Reply

    This is interesting stuff. I use a similar method, though I put Importance first. I figure, if a page has no real business importance, I'll never run a test on it regardless of how much potential for improvement it has (i.e. how badly the page sucks) or how easy it would be to run a test.

    Then again, PIE is a much better acronym than IPE!

    There's a big subjective element to all this. But by attaching numbers to each parameter, putting it all in a matrix, and seeing which page rises to the top, it really does make it easier to decide where to start, and to develop an overall plan. Excellent post!

  3. sheriffbradshaw May 24, 2013 at 8:01 pm - Reply

    The Prioritization for our company would be the biggest impact to our business objectives. The business objectives could vary from department to department so this success metric varies when doing Conversion Optimization.

  4. OCP May 31, 2013 at 8:13 am - Reply

    Isn't the PIE score just an aggregation of a few guesstimates turned into a number out of ten?

    • @chrisgoward June 1, 2013 at 8:05 am - Reply

      As with many good strategic frameworks, it's a mix of objective and subjective measures. We take the data we gather through heuristic analysis, voice of customer, web analytics metrics, and primary research to synthesize relative measures. If it was all just data that could be pulled from an analytics report, we wouldn't need talented strategists to do it. We could just write an algorithm! But, it's not this simple.

      • OCP June 1, 2013 at 9:18 am - Reply

        Fair enough! How does the PIE score itself work, then?

        How does:

        P 10
        I 10
        E 8
        = 9.3?

  5. Sanjay June 2, 2013 at 1:06 pm - Reply

    Very interesting way of looking at conversion optimization. But my question is how do you get the number? For example you have said the Home A/B cluster has a potential of 10 on 10 . How did you come to that number? Was it taking into account certain other factors or just a value you would want to associate with the page's potential? If you could break this down a little bit it would be really awesome!

    Cheers

    • @chrisgoward June 2, 2013 at 9:12 pm - Reply

      The number in each column is a relative ranking of each of the test opportunities based on the relative values for each of the factors we consider for them. The exact scoring method depends on each unique business' environment and data available.

  6. Ole Gregersen February 27, 2014 at 12:43 pm - Reply

    Hi Chris

    In the PIE table, how do you create that list of tests. Its no use just coming up with random tests, so the choice of page and what to test on that page must have great impact on the PIE work.

    So how would you describe how to get to the stage, where you actually have the right pages AND the right tests to run on them, like in your table above?

    /Ole

    • chrisgoward February 27, 2014 at 3:11 pm - Reply

      The art of conversion strategy is involved there, Ole. Our strategists consider many qualitative and quantitative factors to come up with the initial opportunity list, as well as dipping into our test results database for similar websites. That's a part of the creativity and experience we bring to the table for our clients.

  7. Ole Gregersen February 28, 2014 at 8:33 am - Reply

    Hi Chris – maybe, this is the secret sauce of Widerfunnel, but if you were to point someone in a company towards sources of inspiration (besides your own book of course), what might that be?

    /Ole

Leave A Comment

WP Socializer Aakash Web