11 ways to stop FOOC’ing up your A/B tests

Reading Time: 7 minutes

Work long enough in Conversion Optimization and you will hear this phrase:

We tried [insert popular a/b testing tool], but there was a latency issue so we stopped testing.

In 95% of cases, by “latency issue” they’re referring to the noticeable flicker or flash of the original version of a website before test changes are seen. It even has its own acronym: FOOC (Flash of Original Content)*. Here’s a beautiful example I created on the WiderFunnel home page:

An example of FOOC I created. This is not how you want to be A/B Testing.

Why does FOOC matter?

According to a team of MIT neuroscientists, the human brain can identify images in as little as 13 milliseconds.

FOOC can take longer — from 100 ms up to a whole second. Your website visitors will notice.

Get the Technical Optimizer’s Checklist

Download and print off this checklist for your technical team. Check off each item and get prepared for smooth A/B testing ahead!

By entering your email, you’ll receive bi-weekly WiderFunnel Blog updates and other resources to help you become an optimization champion.

 

Is that always a bad thing? No, as David Hauser of Grasshopper discovered:

Our A/B testing tool had a bug that delayed the $25 activation fee from being crossed out until a few seconds after the page loaded. This error ended up creating a much larger uplift than having it already crossed out on load, when the bug was fixed. The result now is that the activation fee shows, and then is crossed out after a few seconds.

Sometimes, FOOC is a good thing as seen on the Grasshopper pricing page. Source: Unbounce

That insight came from a lucky side-effect of the FOOC error, but most times it’s not a good thing.

Whether good or bad, you need to get a handle on your FOOC. It hinders your ability to run controlled experiments. If your variation content is appearing with a flicker every time your page loads, how do you know what effect that’s having on your results?

You don’t, unless you isolate the flicker.

Why does FOOC happen in the first place?

All client-side A/B testing tools are inherently susceptible to FOOC.

“Client-side” means that the changes in your test are being applied as your web page loads, through a tag on your website. This is how the most popular tools on the market do it.

AB testing Optimizely snippet A diagram showing how Optimizely’s snippet works. Source: Optimizely

The client-side nature of these tools is what makes them so easy to get started with: a solo marketer has the ability to launch experiments without the need for a development team. It’s what makes the WYSIWYG Visual Editor a reality.

But that selling point comes at a price. For page changes to occur, a couple things must happen: the snippet of the tool must load and the page elements being modified must load. If either takes too long, your A/B test is in the gutter.

Luckily for us all, there are ways around the challenges of client-side tools.

Follow the eleven tips below, and even if you’re a noob jQuery dabbler, you’ll be able to launch FOOC-free experiments.

1. Speed up your website

Besides being one of the proven ways to increase conversion rates, speeding up your website is a first step in helping prevent flickering or long waits during A/B tests. My favorite tool for this has always been WebpageTest.org. Simple, free, effective. Have your front-end development team look into some of the issues and track performance over time.

abtesting-fooc-5Continue to check your site over time, as small changes can have a big impact on speed.

2. Put your snippet where it needs to go

I’ve seen snippets in footers and I’ve seen them served via Google Tag Manager. Don’t do either. For example, Optimizely’s needs to go as high up as possible in the <head>.

abtesting-fooc-6Whenever possible, move your snippet up to the top of your <head>, assuming you have trimmed jQuery in the snippet.

The drawback is that, yes, Optimizely will be adding a few milliseconds of load time to your pages when loaded for the first time. We haven’t found it to be an issue unless the remaining suggestions aren’t followed.

3. Reduce the size of your snippet

Archive any paused experiments and draft experiments that you don’t need the preview links to and load only a trimmed version of jQuery (this is especially important when loading your snippet at the top of your <head> tag). This will reduce the size of the snippet being loaded on your website, mostly affecting first time visitors.

abtesting-fooc-7

Archive those experiments taking up space in your snippet.

4. Roll up hotfixes

If you’re using your testing tool as a way to make fixes to your website, roll those changes up into project code rather than running them in a separate experiment. If you’re one of many who don’t have access to project-level code, then implement that code along with your current experiment.

abtesting-fooc-8Put “hotfix” code into your project code rather than in an individual experiment.

5. Order your variation code to match your website code

If you’re changing something at the top of your web page, position that change at the top of your variation code. jQuery waits until it finds the element on the page to make the change. If that element comes earlier than later, it will move on to the next line.

This way the content at the top of your website gets changed as quickly as possible.

abtesting-fooc-9

If using jQuery in your variation code, order it so that you’re making changes in the same order that elements load on your website.

6. Consolidate your variation code

If you want to up the size of your headline and change the color, do so in one swift line. If you decide later that you want to reduce the size of the headline, update your existing code rather than adding another line of code to make the reduction in size.

abtesting-fooc-10Group changes into one and remove unused changes.

In conjunction to consolidating code, when making changes via the Visual Editor, keep the scope of your changes to the most specific HTML element possible. Rather than selecting “#mainbody” to modify the attributes of a sub-element, select that sub-element to begin with.

7. Temporarily hide the <body>

No matter how fast your website is, if your original content is loading before your variation code has time to run, you will experience FOOC. To get around this, you’ll need to quickly hide, then show the <body> of your page.

In your experiment-level JavaScript, force Optimizely to run the following:

abtesting-fooc-11Hide the body of the page as quickly as possible by forcing it at the experiment-level.

This hides the <body> as fast as possible, assuming you’ve placed the snippet at the top of the <head>. Then, in your variation code, put a fail-safe (say 3 seconds) to show the body again if something goes wrong.

Insert your variation code after that.

Finally, make the body visible again. Note the 500 millisecond timer on this one. Keep it as low as possible, just enough to avoid a flicker. After all, FOOC is still better than a really slow loading website.
abtesting-fooc-12

Be sure to customize your timers to make sense for your website and the test you’re running.

This gets rid of any flashing of original content (assuming your snippet is not loading asynchronously or too late on the page). The potential drawback is a perceived slowness of the website on first load. That’s why you set a timer to make sure the body is shown before a set threshold.

8. Learn front-end development fundamentals

For those of us who never made it past the “Hello World” lesson in JavaScript 101, it’s a good idea to round out your front-end development knowledge. You don’t need to become a coder, you just need to be able to understand how it works.

It takes no more than a weekend to learn the basics of HTML, CSS, JavaScript and jQuery — the building blocks of DOM manipulation. Head to a free (and fun!) resource like Codecademy to get started.

abtesting-fooc-13Brush up on your front-end development.


Starting here, most of us will need a front-end developer’s help (I’ll admit, I got help from our dev team for this part). If that’s not an option, don’t worry: with the tips above, you should be able to launch FOOC-free tests. Like this article so far? Let me know! 


Now on to steps 9 through 11:

9. Use CSS as much as possible

By default, Optimizely and VWO visual editors produce your edits via jQuery, even for simple things like changing colors. CSS is a faster way to go, whenever feasible.

Instead of this:

abtesting-fooc-14
Do this in the Edit Code window:

abtesting-fooc-15
And add this to the Experiment CSS (or an external stylesheet):

abtesting-fooc-16
10. Cache your selectors

The DOM is slow. To avoid making redundant calls to it, store selectors you’ll be re-using as jQuery objects. In the example below, 3 changes are being made to the same selector.

abtesting-fooc-17Cache selectors you’ll be re-using to avoid going back into the DOM.

11. Code your variations in raw JavaScript

A/B testing visual editors spit jQuery into the code window. jQuery was created to overcome cross-browser issues and save development time. It’s a library of JavaScript shortcuts.

To change the background color of an element in jQuery it goes something like this:

$('.cta').css('background', 'red');

Now the same thing in raw JavaScript:

document.querySelectorAll(".cta")[0].style.backgroundColor = "red";

While the development time savings is significant, it comes at a cost. As with any JavaScript library, jQuery code runs slower than raw JavaScript.

How much slower? Depends on what you’re doing. Without much digging, I found a non-scientific test that resulted in a difference of 60x in performance between jQuery and JavaScript. It’s not significant evidence, but it points to potential speed gains.

Coding some or parts of your variations in raw JavaScript also means that your dev team will have to put in extra time to produce your A/B tests. You’ll want to strike a balance between improving code efficiency and productivity. For more on the topic of JavaScript and jQuery, I urge you to check out this very informative thread on StackExchange.

If you’re using one of the newer schmancy JavaScript frameworks, there are options for writing variation code. Here are some resources to help:

Watch the accompanying Opticon presentation here.

  • Optimizely has published quite a bit on how to deal with sites using angular or other single page app-type situations.

Are those all of the ways to reduce the chances of FOOC? Certainly not. Feel free to add suggestions or questions in the comments below. We can make this an AMA of sorts, regarding FOOC.

FAQs about FOOC

  • Can I use asynchronous loading to avoid FOOC? You can try, but it probably won’t work. Asynchronous loading addresses a separate issue: helping with overall site speed, not FOOC. Given the speed of modern CDNs, snippets loading synchronously should be the least of your concerns. But, if you’re like our neighbors here in Vancouver, PlentyOfFish, with a bajillion users hitting their site at the same time, you may want to be considerate of what and how things load on your pages.
  • Can I use a server-side / proxy testing tool to avoid FOOC? You could, but say good-bye to most of the benefits of a client-side tool.
  • I noticed a major slow down when I added XYZ A/B testing tool on my website. Should I switch to a more popular tool like Optimizely or VWO? Perhaps. There are some tools out there that don’t use distributed CDNs and that include jQuery by default in their snippet. Yes, some will slow down your website.

PS: If you’re an Optimizely power-user, consider checking out a project by WiderFunnel Labs, Liftmap, a great way to increase your A/B testing velocity by managing your CRO workflow.

* As opposed to Flash of Unstyled Content, which refers to a separate problem, usually unrelated to A/B testing.

Enjoy this post? Share with your friends and colleagues:

Private: Alhan Keser

Alhan Keser

Optimization Strategist

Alhan was the head of WiderFunnel Labs and former Director of Optimization Strategy. He is currently Director of Digital Strategy at Blue Fountain Media.

  • jlinowski

    Great stuff. We do a similar thing related to hiding the body tag. We
    sometimes tag the affected content containers with IDs, and hide those
    instead. At other times, we also hard code the variation’s content into
    the control and hide it with CSS as well. Then we hide/show with JS in
    the tool of choice. Cheers.

    • Oh for sure, hard coding is THE way to go if you can. How did I not think of that!? I guess it defeats the purpose – to some degree – of having a “hands-off” testing tool.

      As for delaying the load of the specific containers – it’s a toss-up for me. If they’re in the line of sight as the page loads, hiding them might draw more attention towards those elements as the page loads, resulting in skewed results. But maybe I’m being paranoid. At the same time, who knows what effect hiding the body is having? We do run tests to double-check that the body hiding is not affecting results, but those happen at set intervals, not each test.

      Thanks for the great additions to the list!

  • Sandeep Shah

    I hate the concept of “All client-side A/B testing tools are inherently susceptible to FOOC.” – don’t believe in it whatsoever.

    I’ve been responsible for the launching of 1000s of Projects across the largest websites in the UK, and have never willingly launched tests where FOOC is possible (let alone prevalent). It’s avoidable, in it’s entirety, if vendors care enough to stop it (and clients are capable of best-practise tag integration). The misconception is that this is normal hurts our industry, and allows people to be slack with the solutions they put out into the market.

    If someone says this is absolutely normal and unavoidable, look elsewhere. They’re not doing your website any justice.

Recommended

Explore all blog posts