< Back to main blog page

8 Key Conversion Optimization Strategies to Ensure You (Actually) Test What Matters

Words by: Chris Goward & Chris Goward Last updated: August 27th, 2020 6 min. read

From time to time we invite thought leaders to contribute to this blog. This week, we are honored to bring to you Kevin Hillstrom’s insightful – and sometimes controversial – approach to Conversion Optimization testing.

Kevin Hillstrom

 Kevin Hillstrom is President of MineThatData, where he has worked with sixty-five e-commerce, retail, and catalog executives to understand the relationship between customers, advertising, products, brands, and channels.  Prior to founding MineThatData, Kevin spent nearly two decades in various leadership roles in the retail industry, including Vice President of Database Marketing at Nordstrom, Director of Circulation at Eddie Bauer, and Manager of Analytical Services at Lands’ End.  Kevin has written numerous books about marketing, advertising, and customer behavior, and his MineThatData Blog is one of the most widely read Marketing and Analytics Blogs in the United States.  Kevin can be contacted at [email protected].


Testing is really important!  But, surprisingly, few people execute tests.

And, maybe worse, few people test meaningful activities.

Office politics make a huge difference.  We sit in meetings with Marketing, Creative, and our Information Technology team.  Marketing wants to test a new home page design, Creative claims ownership of the home page (“it’s our job to protect our brand, we can’t have any willy-nilly home page strategy”), and Information Technology controls resources (“we have to prioritize what is important, and this isn’t important, our conversion rate improved from 3.2% to 3.3% last year”).

As a result, we test things that are “easy”.  We test a yellow button against a purple button, we notice a 22% improvement in conversion rates when using a purple button, and we claim victory!

Except, on an annual basis, we seldom if ever observe a 22% increase in conversion, do we?

When we test things that are easy or ephemeral, we don’t get the long-term results we hope for.  In other words, we have to make changes.  We have to test things that matter.  Let’s look at eight testing topics that “matter”!

Testing Strategy #1 = Merchandise:  When customers visit your website, they are generally there to buy something.  So why not test the merchandise you offer a customer on your home page?  Focus on the merchandise that drives conversion.  I frequently look at what are called “order starters”, the items that most frequently appear first in the shopping cart or appear first in a customer order.  Test these items against the items your merchants or creative team would like to feature.

Testing Strategy #2 = Design:  We like to test things that are easy, things like “20% off” against “10% off plus free shipping”.  This is easy; this doesn’t require assistance from any other department.  However, design matters!  Test how you present merchandise to a customer.

Testing Strategy #3 = Personalization:  This doesn’t have to be the uber-personalization that the CRM folks advocate!  Instead, test merchandising strategies that are congruent with customer interests.  I used to work at Nordstrom.  We had customers who bought Cosmetics, and we had customers who purchased Men’s Ties.  You don’t necessarily have to offer the Men’s Tie buyer Cosmetics!

Testing Strategy #4 = Product Density:  So often, we test imagery, and for good reason … it’s easy to test!  Try something different, test product density.  You could offer the customer three items on the home page, or you could offer the customer thirteen items on the home page.  Yes, I realize there is somebody in your company that wants to “protect your brand”.  There’s probably somebody else in your company (the CFO) who wants to protect your brand in a different way … she wants to protect it by generating profit!  If you give the customer more opportunities to buy merchandise, you’ll sell more merchandise.

Testing Strategy #5 = Test Something Different:  This sounds pithy, but it is important.  Your Creative Team might partner with you by offering you the opportunity to test three different home page or landing page designs, designs they crafted.  Try testing something they didn’t craft, against something they did craft.  Either way, you win!  If their tactics work better, then you proved that they are experts, and you can quantify what their expertise means to the bottom line.  If your tactic works better, then you proved that changes are necessary within your presentation strategy.

Testing Strategy #6 = Test Annual Loyalty:  One might think that measuring the performance of different strategies via conversion rate is appropriate.  Unfortunately, it isn’t always appropriate.  When I worked at Nordstrom, I knew that my most loyal customers purchased once a month.  I also knew that my most loyal customers visited our website three times a month.  In other words, on an annual basis, the customer was visiting thirty-six times while purchasing twelve times.  If you implement a strategy that causes the customer to visit twenty-four times while purchasing eleven times, you’ve significantly improved the conversion rate while damaging your brand by nearly ten percent!!  We make these mistakes all of the time!  Measure longer-term conversion (monthly, quarterly), you are likely to observe different outcomes then when you measure conversion rate.

Testing Strategy #7 = Test Strategies That Increase Profit:  If you asked 100 marketers to calculate profit, how many would be able to?  We have a responsibility to test strategies that increase total company profit.  Sometimes, 20% off plus free shipping increases profit, sometimes, 20% off plus free shipping significantly damages profitability.  We need to move beyond measuring conversion rate, we need to instead measure profit.  Our tests must inform us of the most profitable decisions, not the decisions that yield the best possible conversion rate.

Testing Strategy #8 = Re-Test:  If we need to get “technical”, your test results are only applicable to the audience that participated in the test, across the merchandise offered in the test, in the timeframe the test was executed.  This means we make a ton of mistakes!  A certain landing page that performs well in August is not necessarily likely to perform well in January.  Re-test strategies, regardless of statistical significance.  If you can prove that the same test works twice, you’ve proven that the tactic is meaningful.

As eCommerce marketers we can accomplish so much!

But we need to shift our focus from the easy stuff toward strategies and tactics that yield profitable outcomes.

Give these eight strategies a try, let us know what you learn!

Subscribe to get experimentation insights straight to your inbox.