< Back to main blog page

A/B Testing During Black Friday Promotions

Words by: James Flory Last updated: May 31st, 2022 7 min. read

  • A/B Testing
  • Conversion Rate Optimization

Testing during the holidays – BJ Fogg Model

Black Friday, and it’s counterpart Cyber Monday, are some of the largest consumer transaction days for the American economy. Shoppers come out in droves to either complete their holiday shopping, upgrade large ticket appliances/electronics, or just take advantage of great deals and promotions. Those of us who have ventured to participate in Black Friday or Cyber Monday have experienced long lineups in every part of the retail process, and even worse: slow and buggy websites.

However, many of us are so motivated by the fear of missing out on a great deal that we are willing to put up with seemingly anything to check off our holiday shopping list! Let’s take a closer look at why this happens, and how companies can build a testing and shopping experience around it to better understand their shoppers and lift conversion rates throughout the year.

The BJ Fogg Model: Understanding consumer behaviours to test better

As previously discussed in our article A/B Testing During the Holidays, the BJ Fogg model can help companies understand persuasive design and user behaviour. The three parts of this model are “Motivation” up the y axis, “Ability” across the x , and an action line curve in the model as illustrated below:

What the BFM tells us is that:
  1. If you have low motivation to do something, but “easy to do”, there is a strong likelihood that you would do it
  2. If something is ”hard to do”, but you are highly motivated to do it then you are also likely to do it

What we help our partners do with experimentation, is take the baseline point which sits somewhere in the space along the axis of the Fogg model, and move the desired action above the action line, it is here where the user is most likely to convert and complete their checkout journey. If the desired action remains below the action line, then the shopper is much less likely to convert.

How do we do this? We can think of the optimizations we make to a website as focused on improving a user’s motivation to convert, ability to convert, or both.

Hypotheses aimed at improving a user’s ability to convert may be focused around reducing steps in the funnel or process, improving product discoverability, or increasing ease of use on the site.

Alternatively, hypotheses aimed at improving a user’s motivation to convert may be focused around improving the value perception of the product, presence and depth of product information, or – you guessed it – sales (an urgency tactic).

Externality and the Bj Fogg Model: Capturing the Boost on Black Friday

Events like Black Friday are unique in that they cause a dramatic shake-up in both user motivation & ability, allowing an opportunity to convert individuals who may otherwise be below the action line.

We can see below the substantial shift in the model, with shoppers becoming highly motivated during the Black Friday period, however the ability to shop during this period drops significantly. This can be because of long lineups at crowded stores, or websites experiencing an influx of traffic that makes them slow and unresponsive.

The focus here should be on increasing the shoppers ability to purchase and not their motivation, as their motivation is already at very high levels. We can see in the graphic below that increasing the ease in ability is most likely to make the shopper convert, while focusing on motivation will still not yield any conversion. 

Black Friday and Cyber Monday can sometimes see an expected 173%-380% increase in sales and completed transactions. A massive increase no retailer can afford to ignore! As such, many retailers with an e-commerce presence will have developed plans for Black Friday, whether it be items they wish to feature or customized items stocked directly for the event.

As a result, users are flocking to shop online with a heightened motivation to purchase. They likely have a product (or a list of products) they’re looking to shop for and while price will be the key driver of their purchase decision, the ability to purchase will also play an important role.

Historically, some companies might be hesitant to test during Black Friday — the reason being they don’t want to “break something”, and consequently risk losing customers because of a sub-optimal process. Some companies will institute a “code freeze” on Black Friday/Cyber Monday, where no new code is to be added to a site at all for a period of 1-2 weeks around the actual date.

With more mature experimentation programs, we have seen companies capitalize on Black Friday to run tests that help them gain critical new insights into their customers that pays dividends over the long term.

While it’s entirely reasonable to acknowledge the potential risk of testing through the Black Friday period as we have already mentioned, there also exists a large opportunity cost if choosing not to test during this time.

By not experimenting on this surge of traffic and conversions, you’re not only missing a high volume of users to test on, you’re also essentially making a bold statement that the version of your website in its current state is the most optimal and performant version of your site – which as optimizers we know is never the case!

While there is a risk of running a test and sending traffic to a sub optimal Variation A or Variation B, there is also the risk that your control IS the sub optimal experience and you’re missing out on the big gains throughout the year by not capitalizing on the opportunity of Black Friday testing.

So how do we mitigate the risk? Companies can focus on testing methodologies like a Multi Armed Bandit (MAB) through this time period. A MAB automatically diverts traffic to the better performing variation in your experiment. You gain fewer insights, because traffic is automatically diverted to the best performing variation, however revenue generation is maximized..

If you choose not to test during the Black Friday/Cyber Monday shopping period, you can still take this time to learn about users through qualitative research. Interviews can be conducted at a later date with some of the shoppers that chose to visit your site during the Black Friday/Cyber Monday period for further external insights.

Customer Insights: Introducing new Shoppers to your Brand

Customer insights can help you develop future tests and even help inform brand strategy. Many shoppers will be newly introduced to your site during the uptick in visits during Black Friday and Cyber Monday. Conducting experiments and testing during that time period can help you gain new insights about these users with the potential of turning them into repeat customers.

Ready to learn more about how Widerfunnel can help you with your CRO efforts, including testing during the holidays? Get in touch with our team today.

Subscribe to get experimentation insights straight to your inbox.