How long will it take to get conversion optimization results with Google Website Optimizer?
The duration of a test is one of the most frequent questions we hear from our clients. It can be frustrating to see the time remaining estimate fluctuate, or worse, steadily increase as your test progresses.
Unfortunately, there is no simple answer.
The number of test combinations and traffic level are the variables that you can calculate. And Google has provided a calculator that does a ballpark job for you.
However, there are variables to reaching statistical significance that can’t be predicted, which are:
1. The conversion rate of the control
2. The spread between the conversion rate of the control and the alternative/test
If the conversion rate is high and the spread between the control and the variations are large, the test will complete relatively quickly. If, however, the test alternatives are very similar in conversion rates to the control, it will take much longer to reach statistical significance.
That is why our advice to clients is to let us design dramatic tests rather than piddling around with a word here and there. There’s much more upside to be had in being bold with the variations.
It’s also tempting to stop an experiment before it’s complete. And that’s fine to do if it’s not completing fast enough and you want to modify it and try again. But be careful in taking learning from an incomplete test. It’s easy to infer too much into all that green on the bar when it is really not ready to tell you anything.
