< Back to main blog page

October 2021: Experimentation News Round Up

Words by: Daniel Molina Last updated: November 9th, 2021 10 min. read

Experimentation News

The Art of Possible – A Disney Technology Blog

Universal Holdout Groups at Disney Streaming | By Tian Yang

In this blog post, the Disney Streaming team talks about their experience piloting “holdout groups” for Hulu subscribers. Learn more about what a holdout group is, why they needed it and the lessons learned.

Read more

Interpreting A/B test results: false positives and statistical significance

Netflix Technology Blog | By Martin Tingley with Wenjing Zheng, Simon Ejdemyr, Stephanie Lane, and Colin McFarland

Experimentation team at Netflix continues their multi-part series on how Netflix uses A/B testing to inform decisions and innovate. This is Part 3, where they focus on “building intuition” of what a false positive is.

Read more

Interpreting A/B test results: false negatives and power

Netflix Technology Blog | By Martin Tingley with Wenjing Zheng, Simon Ejdemyr, Stephanie Lane, and Colin McFarland

This is part 4 of the multi-part series on A/B Testing by the Netflix experimentation team, focusing on false negatives and statistical power.

Read more

Behavioural Science in the Sign-up Flow

Gousto Engineering & Data Blog | By Kai Mackie

In this blog post, Kai Mackie, Product Analyst at Gousto, shares how Behavioral Science plays a role in a/b testing at Gousto. Some real-world examples include leveraging social proof and loss aversion.

Read more

Personalisation journey: going from a large to small team

Adevinta Tech Blog | By Jordi Mas

Jordi Mas, Engineering Manager at Adevinta, shares his experience with scaling down the team to operate more efficiently, using a data-driven approach. Follow their story on how they took a team of 14 and divided it into smaller teams, leveraging A/B testing to compare results. Adevinta seems to have a very strong culture of experimentation, where they apply the same mindset to overcome their own internal operational challenges.

Read more

How A/B testing can (and can’t) improve your headline writing

NiemenLab | By Nick Hagar and Nicholas Diakopolous

Many news organizations like the New York Times or The Boston Globe are A/B testing their headlines. So NiemanLab partnered with Chartbeat, an analytics tool that offers A/B testing, to analyze 141,000 headline A/B tests run by 293 news websites. Find out what they learned.

Read more

Unlocking digital potential through experimentation and personalisation to drive better outcomes

New Digital Age | By Justin Anovic

The Chief Product Officer of Optimizely, Justin Anovic, shares his perspective and advice on personalization and experimentation. Includes a story on how AeroMexico was able to personalize their check-out experience to boost conversion rates

Read more

Optimizely Survey: Consumers Report Heightened Expectations for Digital Experiences this Holiday Season

Yahoo! Finance | By Business Wire

Optimizely revealed results from a new survey of 5,000 consumers in the US, UK, Germany, Australia and Sweden. Some of the findings include that, on average, American consumers expect their holiday spending budgets to increase 25% compared to last year. Great news for retailers!

Read more

Who’s Hiring?

Ecom Growth Optimization Manager| adidasPortland, OR

Design Lead| WiderfunnelVancouver, BC

Digital Optimization Strategist| WiderfunnelVancouver, BC

Digital Programs Manager| WiderfunnelVancouver, BC

Specialist, Digital Optimization| WileyRemote, USA

Principal Analyst, Experimentation| Cars.comChicago, IL

Site Optimization Lead| LogitechNewark, CA

Software Engineering Manager – Experimentation Platform| Sony Interactive Entertainment PlayStation San Francisco, CA

Manager, UX/UI Designer| NRGHouston, TX

Sr. Marketing Manager, Experiments| AmazonSeattle, WA

Senior Product Manager, Experimentation| ZipRecruiterAustin, TX

Director of Product, Experimentation| NikeBeaverton, OR

Director of Experimentation Engineering| NikeBeaverton, OR

Experimentation Lead, Web Presence| StripeNew York, NY

Experimentation Advocate| RobinhoodMenlo Park, CA

Lead, Experimentation| Macy’sNYC, NY

Sr. Product Manager, Experimentation| WarnerMediaCulver City, CA

Director, Experimentation| ComcastPhiladelphia, PA

Technical Director – Experimentation| Electronic ArtsRedwood City, CA

Manager, Testing Optimization| LionsgateSanta Monica, CA

Optimization Manager| FenderLos Angeles, CA

Subscribe to get experimentation insights straight to your inbox.