Editor’s Note: We just published our 5 favorite ‘A-ha!’ moments from 2016. >> Check ’em out here! <<
Who doesn’t love a story of redemption?
Annie Selke is a multi-faceted company that sells home-ware goods, including home textiles, rugs and luxury linens.
We worked with the Annie Selke team over the course of a year to improve conversion rates across their site (which includes five different brands).
Part I: The ‘losing’ test
We ran an experiment on product category and search results pages that featured a left-hand filter. We had tested this filter previously and knew that there was some sensitivity around its structure.
We wanted to explore further.
In our first variation, we isolated a single element, changing the tiled ‘Sort by’ functionality to a more standard drop-down format.
The second variation was built on the first, but with it, we collapsed the left-hand filter to show only the top four search options.
Both of these variations lost to the control page.
Part II: The turn-around
We had two losing variations on our hands.
But because this test had been structured using isolations, we were able to walk away with a valuable insight:
When we looked at the results by isolation, variation B (changing the filters), which was built on variation A (changing the sort by function), showed the potential for an approximate 20% lift in conversions.
We had also collected some compelling heat map data during the previous experiment that showed users pay particular attention to the word ‘color’.
We went back to the drawing board and designed a new experiment based on these insights.
Variation A retained the collapsed left-hand filter from the previous experiment’s variation B. But, we built the new variation on the original control page, eliminating the losing ‘Sort by’ element.
The final variation was built on variation B, but we made the side filter sticky as the user scrolled down the page.
The winning variation (yes, this test saw a clear winner) lifted sales by 23.6%.
Our ‘losing’ experiment revealed an insight that led us to a huge winner, underlining the importance of experiment design.
The original test may look like a loser, but it’s actually a winner. That’s the power of isolations and the reason you shouldn’t always strive for big changes: when you make a lot of changes at once, you don’t know which change is causing the results.– Michael St Laurent, Optimization Strategist
This post is second in a 5-part series. If you missed our first ‘A-ha!’ moment, check out the link below:
Stay tuned for more of our favorite ‘A-ha!’ moments in the coming days.
Subscribe to get experimentation insights straight to your inbox.