Editor’s Note: We just published our 5 favorite ‘A-ha!’ moments from 2016. >> Check ’em out here! <<
A brief introduction
DMV.org is a non-government content-based website. They provide visitors with information that they might need for a DMV visit in each of the 50 U.S. states. The company earns revenue through performance-based advertising on their content pages.
The site spans thousands of pages of information.
Because we’ve been working with DMV.org for several years, we were able to pitch the following design test for their auto insurance conversion funnel (and they didn’t laugh in our faces!)
The utlimate mini-banner showdown
When users on DMV.org are looking for insurance rate estimates ― and thereby entering the auto insurance conversion funnel ― they are prompted to enter their zip code into a mini-banner. Once they’ve entered their zip, they are shown a list of auto insurance providers in their area.
We wanted to isolate various elements on this mini-banner to see if we could increase conversions (in this case, the product click-through rate) with slight, or not-so-slight, design changes.
The control banner featured a small car crash image:
In our variations, we decided to change things up a bit.
We tested a different image of a car crash in one variation and an animated GIF of a fender-bender in another. We tested a photo of a sad-looking driver in one and ― at the request of our pet-loving CEO ― we tested an image of a driving dog in another.
With our final variation, we tried something a little nuts: we changed the shape of the banner completely, making it the silhouette of a car.
Our design team hated this variation.
My initial reaction was ‘Oh no.’ I didn’t think it was a bad idea per se, I just thought it was so ugly.– Jules Skopp, CRO Experience Designer
As you might’ve guessed (given the build up) the car-shaped mini-banner walloped the competition.
Much to the dismay of our design team and the surprise of our strategy team, the little car-shaped banner that could increased the product click-through rate by 89.86% (compared to less than 10% for the other images) which led to a revenue per visitor increase of 74.92%.
We had an office-wide poll going to see which banner would win. Nobody chose the car.– Nick So, Optimization Strategist
The design of this banner was totally out of left field, so much so that members of both teams thought it was a waste of time to test. But users loved it!
We had a million questions: Did users feel that the car-shaped banner was more relevant somehow? Or did the atypical design and shape simply make this banner more attention-grabbing?
This result spawned some crazy questions: What if the car was shaped like a Ferrari? Would different car shapes perform better in different states? The possibilities are endless.– Nick So, Optimization Strategist
Most importantly, this test reminded us that our opinions don’t matter. Users will sometimes behave in completely unpredictable ways – if you have a gut feeling, test it.
The data doesn’t lie.
This post is fourth in a 5-part series. If you missed any previous ‘A-ha!’ moments, check out the links below:
- A-ha! Personalization goes beyond design
- A-ha! Isolations turn a losing test into a winner
- A-ha! The (social) proof is in the video
Stay tuned for our last post in this series!
Discover how your experimentation program stacks up!
Benchmark your experimentation maturity with our new 7-minute maturity assessment and get proven strategies to develop an insight-driving growth machine.Get started