Find out how BuildDirect increased monthly revenue by $1 Million per month while at the same time improving design and usability of their website. Chris Goward, CEO of WiderFunnel, will present a detailed case study that shows what was tested, the process, the challenges and the amazing results.
Watch the video to learn:
- How the LIFT Model™ works in practice
- How the strongest hypotheses were generated
- Real winning and losing A/B/n test variations
- How an ongoing conversion strategy led BuildDirect to abandon traditional web design
Chris Goward was one of the first people to look at online content and say: “We should test that!” From that revelation he founded WiderFunnel – the full-service marketing optimization company that pioneered landing page and conversion rate optimization methods for companies such as Google, Electronic Arts, Citrix and BabyAge.com.
He is the brains behind the LIFT Model™ and Kaizen Methods™ which have helped companies improve their online results by up to 400% for lead generation and e-commerce sales.
Chris speaks at conferences and seminars around the world to evangelize how marketers should test their marketing to get more leads, sales and revenue. He is author of the new book, “You Should Test That!” published by Wiley.
Question and answer
Please note, this transcript has been edited for readability and are not Chris’s responses verbatim during the Q&A portion of the webinar.
Does only providing better copy matter? In your examples you have focused on visuals as what matters.
Visual changes certainly aren’t all of the tests we run. In fact, some of our biggest wins have been from copy changes. At WiderFunnel, we try to understand the triggers that move users into action, we use these insights to make copy changes that clarify a company’s value proposition. For example, we recently tested adding a tagline to one of our client’s logos. The change clarified their value proposition, resulted in a massive win site wide, and an embarrassing amount of additional revenue for the client. That 4-word tagline is now a critical component across all of the company’s marketing activities.
There are many different approaches to copy, from copy changes on buttons or headlines, to re-writing entire pages that can drive conversions. At WiderFunnel, often our copy experiments will test a variety of options against each other – for example short pages versus a story based approach, or versus a testimonial based approach, or sometimes versus a long copy feature based approach.
Referring to the BuildDirect example, was your conversion rate goal the same for all of the tests you ran across the different webpages?
In the particular example from the webinar – yes.
At the strategy phase, which is the first part of our engagement, we solidify our understanding of our client’s revenue improvement goals. In BuildDirect’s case, there was more than one goal, so we took the mixture and came up with a blended metric. The purpose of this is to make sure every test drives the same conversions. It also ensures that we are driving results as close to revenue as possible (not just a step toward revenue, which can lead to misleading results). We often create blended metrics for clients that have more than one goal, for example multi-channel conversions, competition conversions, or different types of products.
If you would like more information on setting goals for Conversion Optimization, I have a blog post called ‘Don’t Try to Lower your Bounce Rate‘.
How do you maintain a line of testing, using the PIE Framework, without losing your organizational attention span?
Maintaining organizational attention span can be a challenge. Especially if your alone in your efforts to promote testing and not immediately getting the LIFT‘s you need to get your co-workers excited.
If you don’t start with a rigorous process in place, your early mistakes can dry out all of your scarce organizational momentum too quickly. This is one of the reasons bringing in an expert can help – you will know you have the process right, and will be backed by third-party credibility to help create momentum and excitement within your organization.
Are customers always segmented based on device usage? Or do you use other ways to segment?
Device usage is only one method for segmentation. There are many other methods you can use to segment, such as traffic source, or tiers (meaning your value of the customer). A lot of times, how we segment is unique to the client; it is based on what data points we can gather, what our client’s business model is, and what their segmentation hypotheses are.
Segmentation ideas can come in two ways. The first, is creating a segmentation hypothesis and validating it through testing. The second, testing on a larger, more homogenous amalgamated segment, and using the test results to find out if there are valid segments that respond differently to the different variations. Both are great ways to segment. Using one or the other depends on your confidence in your pre-defined segments and your segmentation hypotheses.
Maintaining multiple streams of tests requires resources and therefore comes at a cost. Because of this, it is important to test to find which segments are actually important to test within.
If you would like more information on segmentation, my blog post ‘8 Steps to Amazing Website Segmentation Success‘, may be able to answer some of your questions.
Do you have an example of where you tried something unusual and had good results?
One of the things we try to do at WiderFunnel is allow ourselves to test ‘Left Field’, or ‘Wacky’ variations. Often it will be these tests that surprise us and lead to some of our most profound insights. For example we have seen that flipping columns (swapping the left and right columns of a page) has converted more leads. This was a surprise to us, and we have since used the insight to inspire subsequent hypotheses and tests for our clients.
What are the best CRO tools you recommend?
Generally, I categorize tools into one of two categories: hypothesis tools, and testing tools.
Hypothesis tools are used to gather information about your prospects. They can be qualitative: used for surveying, or understanding the ‘voice of the customer.’ Or, they can be quantitative, such as web analytics, and can be used to understand how customers are interacting with the website.
The second category are the A/B testing tools. There are all kinds of these out now, including: Optimizely, Visual Website Optimizer, Adobe, Google Content Experiments, AB Tasty, and Marketizator.
How do you quickly deploy redesigns to production given the usual technical complexity of a website?
The complexity of a website’s code determines how easy it is to hard-code the winners. Instead of hard-coding every winning variation, we are able to rapidly test through a series of experiments using an AB testing tool. At periodic intervals, after a series of experiments, we will hard-code the latest winning variation. There is a trade off between the quick load speed of having the page hard-coded right away, versus the time investment and resource allocation of implementation after every experiment. When we are making the decision on when to implement we look at the complexity of the platform, our resource availability to implement, and whether we, or our client, has tools available to make the task easier.
How much traffic is needed on a page to achieve statistical significance when testing? Is there a traffic number that applies to all tests, or does it depend on various factors?
Traffic depends on various factors. The traffic needed to complete a test with statistical significance depends on the volume of traffic to the page, the conversion rate of the page, and the conversion rate difference between the variations.
Despite the amount of traffic being variable, there are ways to speed up your tests:
- If you have a high variance in conversion rate between the variations, your test will complete faster and require less traffic. Testing dramatically is one way to produce variance between variations.
- If you have fewer variations you will need less traffic per variation, and the experiment overall will complete faster.
- If you have more traffic to the page your experiment will, of course, complete faster because you have a larger sample size to work with.
- Segmentation splits traffic, therefore if you have fewer segments, your experiment will complete faster.
As a rule of thumb, we plan our tests based on the traffic volume coming in and schedule them to complete within a two-week window. The strategists at WiderFunnel are very skilled at working out what will have the best return on effort, the greatest insights, and the quickest results.
What is one of the most successful tests that you’ve run? What were the inputs that led you to run that test (or was it just a lucky guess)?
At WiderFunnel, our strategists are running successful tests every day. We publish many of the most successful and interesting ones on our case studies page.
We use a variety of inputs for all of our tests: qualitative data, web analytics, expert heuristic analysis and our database of other test results. Our research plays a big part in our hypotheses creation, but our structured process and LIFT Model are what really drive our results.
What are your CRO favourite resources (other than best practices we all see everywhere)?
The so-called ‘best practices’ you see everywhere are one of the worst places to get ideas. Those are the ideas everyone else is already using. If they really were killer ideas, generating huge revenue, do you think they’d publish them? Nope.
The best ideas come from good data, combined with great intuition, and old-fashioned elbow grease. It comes from combining qualitative data, web analytics, expert heuristic analysis, and our database of other test results into a rigorous process of testing powerful hypotheses.
What changes would you make to your approach to ESR if the site doesn’t get a high volume of visitors?
With lower-traffic websites, more dramatic tests need to be run. The worst-case scenario would be to do a traditional redesign and then test the entire new site against the old one using a subdomain redirect. Put the next design on a ww2.domain.com and test it against the original www.domain.com site. This type of test won’t tell you which changes were positive or negative, but at least you’ll know whether the new site is an improvement overall.
What would you recommend? Focusing on AB testing step by step (sales funnel) until reaching the “Thank you page!” or AB Testing different steps in sales funnel with no specific order?
The question of which step in the funnel to start testing depends on the three PIE Framework factors: Potential, Importance and Ease. Check out the detailed answer here: //widerfunnel.com/conversion-rate-optimization/how-to-prioritize-conversion-rate-optimization-tests-using-pie.