This post, by Jānis Lanka, is first in a series called “The View from the Conversion Trenches” where elite marketers share their perspectives on Conversion Optimization and the resulting organizational impacts.

Vancouver Olympics Store
A/B Split Testing

With the closing of the Vancouver 2010 Winter Olympics, all the medals have been handed out, and all the tourists have fled the city. I was fortunate to have been a part of the Elastic Path Software Team that managed the Official Vancouver 2010 Olympic Store since the store launched in December 2007. So, in the spirit of the Games, I will share a little retrospective on my experiences!

During early days of the online store operations, it had a very limited product assortment. Each week as we got closer to the Games, more and more products got shelved in our warehouse. Eventually, the warehouse grew to almost up to 5,000 SKUs.

With that much inventory to sell in a very, very short period of time, we had to ask ourselves, how do we sell all of this quickly? Is the current customer experience worthy of a gold medal?? Is there a way to increase online revenue?

We knew there were areas of the site that could use optimization: our homepage could be less busy; our product details page could better display key information; our checkout process could be shorter (too many people were abandoning the process mid-way).

So many opinions, so little time

However, since these were “opinions,” and nobody had answers based on facts, we decided to quickly validate the various theories, just in time for the Winter 2010 Games buying season (this deadline was a serious one!).

First we had to decide on which testing tool to use. We selected Google Website Optimizer (GWO) because we expected it would be easy to implement. GWO is a free website testing and optimization tool that would help gather data to reveal which page converted best.

However, we had the challenge of integrating GWO into our backend. Even with our strong team of developers, we had to enlist external help in order to meet the tight deadlines.

We turned to another local company, WiderFunnel Marketing Optimization, for help with our A/B split testing project. They were amazing at walking us through the entire planning, design, and technical implementation process while answering all of our questions. In addition, WiderFunnel interpreted the various data points we had gathered in Google Analytics so we could confidently select the winning page templates.

Going for Gold!

We completed four different A/B Experiments. WiderFunnel provided the conversion analysis, hypotheses, wireframing, graphic design and development in two of those experiments. The other two experiments were done by our own in-house team.

In these experiments, we tackled our four most important problems: checkout process, product details page, homepage, and category page. Each one of the experiments generated improved conversion rates and improved average order value!

In addition to more sales per web visitor and a higher revenue per order level, we noticed another benefit impacting our company, a cultural change. Since we began engaging in testing and optimization, we’ve noticed that opinions expressed in meetings were now being complemented with facts, new feature implementation processes have better structure, and team members had a renewed trust in each other because decisions were now data driven.

When something sparks a debate, someone always pipes up, “Why don’t we test that?”

About the Author
Captivated by innovation, Jānis Lanka (@janislanka) is particularly drawn to fresh thinking in areas such as ecommerce, entrepreneurship, marketing, design, and user interface. Jānis currently heads the front-end taskforce at Elastic Path Software, an enterprise ecommerce solutions provider.