How to prioritize conversion rate optimization tests using PIE

| May 19, 2013

One of the most common questions I’m asked about conversion optimization is:

How do I figure out where to test first?

It’s a smart question to ask. You can’t test all pages at once. With limited time and resources to commit and, most importantly, limited traffic to allocate to each test, test prioritization is an important part of your conversion rate optimization plan.

The alternative happens when companies start with the areas they believe are obvious. You might consider the popularity of the page or look for under-performing pages, or start with pages your HiPPO hates.

Those are all things to consider, but they don’t give you the whole picture.

Optimizing the optimization process is often just as important as the tests themselves. Prioritizing where you invest energy will give you better returns by emphasizing pages that are more important to the business.

The Three Criteria to Prioritize Pages

You need to consider three criteria to prioritize which pages to test and in which order: Potential, Importance and Ease.


How much improvement can be made on the pages? Although I’ve yet to find a page without some potential for improvement, you can’t test everywhere at once and you should prioritize your worst performers. This should take into account your web analytics data, customer data and expert heuristic analysis of user scenarios.


How valuable is the traffic to the pages? Your most important pages are the ones with the highest volume and the costliest traffic. You may have identified pages that perform terribly, but if they don’t have significant volume of costly traffic, they aren’t testing priorities.


How complicated will the test be to implement on the page or template? The final consideration is the degree of difficulty a test will take to get running on a page, which includes technical implementation and organizational or political barriers. The less time and resources you need to invest for the same return, the better. This includes both technical and “political” ease. A page that would be technically easy may have many stakeholders or vested interests that can cause barriers. I’m looking at you, home page.

You can quantify each of your potential opportunities based on these criteria to create your test priority list.

We use this PIE Frameworkâ„¢ in a table to turn all of the data inputs into an objective number ranking.
conversion rate optimization what to test first

Your Prioritization is Unique

There are no standard rules for which pages are best to prioritize. Your website lives in a unique target market, including factors like your competition, seasonality, and internal cultural environment; all of these affect how your site is used and should be optimized. The priority rating you give each of your potential test pages will depend on this unique business environment.

By using the PIE Framework, you’ll remove gut feeling from the decision and focus your team on an objective, relative ranking.

Applying the PIE Framework

Recently, on my morning walk to work, I recorded a story of how we applied PIE with a conversion optimization client.

How do you prioritize your conversion optimization test opportunities?

Add your comment below!

If you find this interesting please share with your friends or colleagues:

  • Rob Kingston

    I normally use a MoSCoW rating alongside Time/Cost and expected uplift.

    I used to score tests based on Ease, Benefit and Cost but the resulting score never worked out all too well. PIE sounds very tasty though.

  • Michael Straker

    This is interesting stuff. I use a similar method, though I put Importance first. I figure, if a page has no real business importance, I'll never run a test on it regardless of how much potential for improvement it has (i.e. how badly the page sucks) or how easy it would be to run a test.

    Then again, PIE is a much better acronym than IPE!

    There's a big subjective element to all this. But by attaching numbers to each parameter, putting it all in a matrix, and seeing which page rises to the top, it really does make it easier to decide where to start, and to develop an overall plan. Excellent post!

  • sheriffbradshaw

    The Prioritization for our company would be the biggest impact to our business objectives. The business objectives could vary from department to department so this success metric varies when doing Conversion Optimization.

  • OCP

    Isn't the PIE score just an aggregation of a few guesstimates turned into a number out of ten?

    • @chrisgoward

      As with many good strategic frameworks, it's a mix of objective and subjective measures. We take the data we gather through heuristic analysis, voice of customer, web analytics metrics, and primary research to synthesize relative measures. If it was all just data that could be pulled from an analytics report, we wouldn't need talented strategists to do it. We could just write an algorithm! But, it's not this simple.

      • OCP

        Fair enough! How does the PIE score itself work, then?

        How does:

        P 10
        I 10
        E 8
        = 9.3?

        • @chrisgoward

          It's just an average of the three.

  • Sanjay

    Very interesting way of looking at conversion optimization. But my question is how do you get the number? For example you have said the Home A/B cluster has a potential of 10 on 10 . How did you come to that number? Was it taking into account certain other factors or just a value you would want to associate with the page's potential? If you could break this down a little bit it would be really awesome!


    • @chrisgoward

      The number in each column is a relative ranking of each of the test opportunities based on the relative values for each of the factors we consider for them. The exact scoring method depends on each unique business' environment and data available.

  • Ole Gregersen

    Hi Chris

    In the PIE table, how do you create that list of tests. Its no use just coming up with random tests, so the choice of page and what to test on that page must have great impact on the PIE work.

    So how would you describe how to get to the stage, where you actually have the right pages AND the right tests to run on them, like in your table above?


    • chrisgoward

      The art of conversion strategy is involved there, Ole. Our strategists consider many qualitative and quantitative factors to come up with the initial opportunity list, as well as dipping into our test results database for similar websites. That's a part of the creativity and experience we bring to the table for our clients.

  • Ole Gregersen

    Hi Chris – maybe, this is the secret sauce of Widerfunnel, but if you were to point someone in a company towards sources of inspiration (besides your own book of course), what might that be?