Work long enough in Conversion Optimization and you will hear this phrase:
We tried [insert popular a/b testing tool], but there was a latency issue so we stopped testing.
In 95% of cases, by “latency issue” they’re referring to the noticeable flicker or flash of the original version of a website before test changes are seen. It even has its own acronym: FOOC (Flash of Original Content)*. Here’s a beautiful example I created on the WiderFunnel home page:
An example of FOOC I created. This is not how you want to be A/B Testing.
Why does FOOC matter?
According to a team of MIT neuroscientists, the human brain can identify images in as little as 13 milliseconds.
FOOC can take longer — from 100 ms up to a whole second. Your website visitors will notice.
Is that always a bad thing? No, as David Hauser of Grasshopper discovered:
Our A/B testing tool had a bug that delayed the $25 activation fee from being crossed out until a few seconds after the page loaded. This error ended up creating a much larger uplift than having it already crossed out on load, when the bug was fixed. The result now is that the activation fee shows, and then is crossed out after a few seconds.
That insight came from a lucky side-effect of the FOOC error, but most times it’s not a good thing.
Whether good or bad, you need to get a handle on your FOOC. It hinders your ability to run controlled experiments. If your variation content is appearing with a flicker every time your page loads, how do you know what effect that’s having on your results?
You don’t, unless you isolate the flicker.
Why does FOOC happen in the first place?
All client-side A/B testing tools are inherently susceptible to FOOC.
“Client-side” means that the changes in your test are being applied as your web page loads, through a tag on your website. This is how the most popular tools on the market do it.
A diagram showing how Optimizely’s snippet works. Source: Optimizely
The client-side nature of these tools is what makes them so easy to get started with: a solo marketer has the ability to launch experiments without the need for a development team. It’s what makes the WYSIWYG Visual Editor a reality.
But that selling point comes at a price. For page changes to occur, a couple things must happen: the snippet of the tool must load and the page elements being modified must load. If either takes too long, your A/B test is in the gutter.
Luckily for us all, there are ways around the challenges of client-side tools.
Follow the eleven tips below, and even if you’re a noob jQuery dabbler, you’ll be able to launch FOOC-free experiments.
1. Speed up your website
Besides being one of the proven ways to increase conversion rates, speeding up your website is a first step in helping prevent flickering or long waits during A/B tests. My favorite tool for this has always been WebpageTest.org. Simple, free, effective. Have your front-end development team look into some of the issues and track performance over time.
Continue to check your site over time, as small changes can have a big impact on speed.
2. Put your snippet where it needs to go
Whenever possible, move your snippet up to the top of your <head>, assuming you have trimmed jQuery in the snippet.
The drawback is that, yes, Optimizely will be adding a few milliseconds of load time to your pages when loaded for the first time. We haven’t found it to be an issue unless the remaining suggestions aren’t followed.
3. Reduce the size of your snippet
Archive any paused experiments and draft experiments that you don’t need the preview links to and load only a trimmed version of jQuery (this is especially important when loading your snippet at the top of your <head> tag). This will reduce the size of the snippet being loaded on your website, mostly affecting first time visitors.
Archive those experiments taking up space in your snippet.
4. Roll up hotfixes
If you’re using your testing tool as a way to make fixes to your website, roll those changes up into project code rather than running them in a separate experiment. If you’re one of many who don’t have access to project-level code, then implement that code along with your current experiment.
Put “hotfix” code into your project code rather than in an individual experiment.
5. Order your variation code to match your website code
If you’re changing something at the top of your web page, position that change at the top of your variation code. jQuery waits until it finds the element on the page to make the change. If that element comes earlier than later, it will move on to the next line.
This way the content at the top of your website gets changed as quickly as possible.
If using jQuery in your variation code, order it so that you’re making changes in the same order that elements load on your website.
6. Consolidate your variation code
If you want to up the size of your headline and change the color, do so in one swift line. If you decide later that you want to reduce the size of the headline, update your existing code rather than adding another line of code to make the reduction in size.
Group changes into one and remove unused changes.
In conjunction to consolidating code, when making changes via the Visual Editor, keep the scope of your changes to the most specific HTML element possible. Rather than selecting “#mainbody” to modify the attributes of a sub-element, select that sub-element to begin with.
7. Temporarily hide the <body>
No matter how fast your website is, if your original content is loading before your variation code has time to run, you will experience FOOC. To get around this, you’ll need to quickly hide, then show the <body> of your page.
Hide the body of the page as quickly as possible by forcing it at the experiment-level.
This hides the <body> as fast as possible, assuming you’ve placed the snippet at the top of the <head>. Then, in your variation code, put a fail-safe (say 3 seconds) to show the body again if something goes wrong.
Insert your variation code after that.
Finally, make the body visible again. Note the 500 millisecond timer on this one. Keep it as low as possible, just enough to avoid a flicker. After all, FOOC is still better than a really slow loading website.
Be sure to customize your timers to make sense for your website and the test you’re running.
This gets rid of any flashing of original content (assuming your snippet is not loading asynchronously or too late on the page). The potential drawback is a perceived slowness of the website on first load. That’s why you set a timer to make sure the body is shown before a set threshold.
8. Learn front-end development fundamentals
Brush up on your front-end development.
Starting here, most of us will need a front-end developer’s help (I’ll admit, I got help from our dev team for this part). If that’s not an option, don’t worry: with the tips above, you should be able to launch FOOC-free tests. Like this article so far? Let me know! Tweet
Now on to steps 9 through 11:
9. Use CSS as much as possible
By default, Optimizely and VWO visual editors produce your edits via jQuery, even for simple things like changing colors. CSS is a faster way to go, whenever feasible.
Instead of this:
Do this in the Edit Code window:
And add this to the Experiment CSS (or an external stylesheet):
10. Cache your selectors
The DOM is slow. To avoid making redundant calls to it, store selectors you’ll be re-using as jQuery objects. In the example below, 3 changes are being made to the same selector.
Cache selectors you’ll be re-using to avoid going back into the DOM.
To change the background color of an element in jQuery it goes something like this:
document.querySelectorAll(".cta").style.backgroundColor = "red";
- The team at Trulia created their own Optimizely / node.js integration. By the way, really nice folks.
Watch the accompanying Opticon presentation here.
- Optimizely has published quite a bit on how to deal with sites using angular or other single page app-type situations.
Are those all of the ways to reduce the chances of FOOC? Certainly not. Feel free to add suggestions or questions in the comments below. We can make this an AMA of sorts, regarding FOOC.
FAQs about FOOC
- Can I use asynchronous loading to avoid FOOC? You can try, but it probably won’t work. Asynchronous loading addresses a separate issue: helping with overall site speed, not FOOC. Given the speed of modern CDNs, snippets loading synchronously should be the least of your concerns. But, if you’re like our neighbors here in Vancouver, PlentyOfFish, with a bajillion users hitting their site at the same time, you may want to be considerate of what and how things load on your pages.
- Can I use a server-side / proxy testing tool to avoid FOOC? You could, but say good-bye to most of the benefits of a client-side tool.
- I noticed a major slow down when I added XYZ A/B testing tool on my website. Should I switch to a more popular tool like Optimizely or VWO? Perhaps. There are some tools out there that don’t use distributed CDNs and that include jQuery by default in their snippet. Yes, some will slow down your website.
PS: If you’re an Optimizely power-user, consider checking out a project by WiderFunnel Labs, Liftmap, a great way to increase your A/B testing velocity by managing your CRO workflow.
* As opposed to Flash of Unstyled Content, which refers to a separate problem, usually unrelated to A/B testing.