Editor’s Note: We just published our 5 favorite ‘A-ha!’ moments from 2016. >> Check ’em out here! <<
A brief introduction
One of our clients this past year is an e-commerce company. They sell a wide variety of beauty products and use a subscription model. Subscribers are part of an exclusive club and receive a box of beauty products each month.
Most of these subscribers come in from a Facebook ad.
We were focused on this client’s 4-step acquisition funnel. After targeting the first step, we wanted to apply what we had learned to the second step of the funnel.
Based on previous tests, we knew that subscribers responded to social proof and were sensitive to both community messaging and the quality of our client’s products.
We ran the following test on desktop and on mobile, segmenting the results by device.
Press play: Round I
The control page for this experiment highlighted the customizable nature of the monthly beauty box.
In one variation, we replaced this information with a video of a subscriber talking about how much she loves being a member of this community. With this video, we hoped to draw out visitors’ appreciation for social proof.
We had actually tested social proof before, but it didn’t work. We used a statement that said ‘Join thousands of other subscribers’ but it sounded so fake. The video added genuineness to the messaging.– Claire Vignon, Director of Optimization Strategy
In another variation, we replaced the control content with a video of the company’s CEO — an already-prominent figure on the their social media pages. In this video, she discussed the community, the company’s core values and the quality of their products.
Both variations saw significant lift in completed orders on mobile: 12.9% and 10.0% respectively.
Press play: Rounds II & III
As is our habit, we decided to build on this test. We adopted the winning social proof variation as our control and created two more variations:
- In the first variation, we added the video of the CEO beneath the subscriber testimonial
- In the second, we shortened the subscriber testimonial from 2 minutes to 1 minute
The variation that included both videos increased conversions by 3.5% on desktop and 6.5% on mobile. The variation with the shorter video actually decreased conversions among desktop users but lifted conversions on mobile by a significant 12.2%.
While mobile users seem to enjoy the video content, their possible limited attention spans could explain the dramatic lift caused by a shorter, more concise video.
Desktop users, on the other hand, may have more time and prefer more information to become familiar with the company’s value proposition.
We kept going.
Our next move was to combine these two variations into one. It featured the shorter video testimonial and the video of the CEO. We saw a 15.5% lift for desktop users and a 12.2% lift for mobile users.
In this series of tests, we found that video can be a powerful social proof tool across all devices.
The client in this case was fearful that mobile users might be too impatient to watch a video. They worried that visitors may not want to waste data, but our results showed these fears to be unfounded.
Our continued testing led us to discover a winning video combination for our client’s users. It also revealed insights about video length on mobile versus on desktop.
This post is third in a 5-part series. If you missed either of our previous ‘A-ha!’ moments, check out the links below:
Discover how your experimentation program stacks up!
Benchmark your experimentation maturity with our new 7-minute maturity assessment and get proven strategies to develop an insight-driving growth machine.Get started