< Back to main blog page

Experimentation News Round Up: Best of 2021

Words by: Daniel Molina Last updated: January 7th, 2022 11 min. read

  • Experimentation News

December is always a great chance to look back on the year and reflect on the highlights. In the spirit of this, here are some of the top blog posts written this year. All of these come from blog posts by employees at top brands, as it’s always insightful to hear how each organization approaches experimentation and their perspective on certain challenges.

From everyone here at Widerfunnel, we wish you all happy holidays and a happy new year.

Experimentation News

[Hulu] How We Scaled Experimentation At Hulu

Disney Streaming Blog | By By Sidhant Bakhru

In this blog post, Sidhant Bakhru, Senior Product Marketing Manager at Hulu, shares details about how Hulu created a Center of Excellence and the key actions the group took to take experimentation efforts to the next level.

Read more

[Financial Times] 6 Lessons from rapid experimentation at the Financial Times

Via Medium.com | By Simi Agbaje

In this blog, Simi Agbaje explains why they decided a rapid experimentation approach — and how it allowed them to validate as many concepts as possible as quickly as possible.

Read more

[HelloFresh] Getting strategic about your A/B tests.

Medium.com | By Rina Fufaeva

In this post, Senior Product Manager Rina Fufaeva explains why experiment velocity is so important to HelloFresh.

Read more

[Vistaprint] Building a Culture of Experimentation

Vistaprint Blog | By Lukas Vermeer

Vista’s Director of Experimentation shares his insights and an outline of the culture they seek to build at the organization.

Read more

[Lyft] Parameter Exploration at Lyft

Lyft Engineering Blog | By Henry Quan

At Lyft, the experimentation team’s purpose is to drive data-driven decision making. In this blog post, Henry explains what parameters are, how they can be optimized, and the challenges they faced.

Read more

[Netflix] Decision Making at Netflix

Netflix Technology Blog | By Martin Tingley with Wenjing Zheng, Simon Ejdemyr, Stephanie Lane, and Colin McFarland

This introduction was the first in a multi-part series on how Netflix uses A/B tests to make decisions that continuously improve their products.”

Read more

[Squarespace] How We Reimagined A/B Testing at Squarespace

Engineering.squarespace.com/blog | By John Absher, Joanna Tang, Srishti Sanya

In this blog post, learn about the tooling and culture improvements Squarespace made and as a result how A/B testing at Squarespace became faster, more accurate, and more impactful.

Read more

[Airbnb] Designing Experimentation Guardrails

Medium.com Blog: Airbnb | By Tatiana Xifara, Reid Andersen, Kathy Yang & Ali Rauh

In this blog post, the Airbnb team explains the Experiment Guardrails framework implemented at Airbnb, and its impact on key metrics when experimenting at scale.

Read more

[New York Times] How We Rearchitected Mobile A/B Testing at The New York Times

New York Times | By The NYT Open Team

The New York Times uses A/B tests to make decisions about the products and features they release, but their mobile test allocation wasn’t separating users properly. Here’s how they fixed it.

Read more

[Disney] The Art of Possible – A Disney Technology Blog

Universal Holdout Groups at Disney Streaming | By Tian Yang

In this blog post, the Disney Streaming team talks about their experience piloting “holdout groups” for Hulu subscribers. Learn more about what a holdout group is, why they needed it and the lessons learned.

Read more

[DoorDash] Running Experiments with Google Adwords for Campaign Optimization

DoorDash Engineering Blog | By Yingying Chen & Heming Chen

In this post, Doordash explains how they were able to run an experiment using the Google Adwords Testing Platform: “Experimentation for digital marketing is all about figuring out the most efficient and effective ways to run marketing campaigns on a digital channel.”

Read more

[Wayfair] Speeding up A/B Tests with Delayed Reward Forecasting

Wayfair Tech Blog | By Le Wang

In this post, Le Wang explains how some experiments at Wayfair can last 60 days or more and what they needed to do in order to speed up learning in experiments while still optimizing for long term rewards.

Read more

Who’s Hiring?

Website Product Owner| Best EggWilmington, DE

Design Lead| WiderfunnelVancouver, BC

Digital Optimization Strategist| WiderfunnelVancouver, BC

Digital Programs Manager| WiderfunnelVancouver, BC

Manager, Experimentation| Canadian Tire CorpOntario, CA

Conversion Rate Optimization Specialist| The Motley FoolUSA

Ecom Growth Optimization Manager| adidasPortland, OR

Senior Backend Engineer, Growth: Experimentation| GitLabVancouver, BC (Remote)

Senior Manager, Conversion Experimentation| Disney StreamingNYC, NY

Director, Growth Marketing| RealmNYC, NY

Senior Software Engineer – Experimentation| Realtor.comVancouver, BC

Sr. Marketing Manager, Experiments| AmazonSeattle, WA

Senior Product Manager, Experimentation| ZipRecruiterAustin, TX

Director of Experimentation Engineering| NikeBeaverton, OR

Experimentation Lead, Web Presence| StripeNew York, NY

Lead, Experimentation| Macy’sNYC, NY

Sr. Product Manager, Experimentation| WarnerMediaCulver City, CA

Subscribe to get experimentation insights straight to your inbox.