< Back to main blog page

February 2022: Experimentation News Round Up

Words by: Daniel Molina Last updated: May 31st, 2022 10 min. read

  • Experimentation News

Experimentation News

How Managers Can Build a Culture of Experimentation

Harvard Business Review | By Frank V. Cespedes and Neil Hoyne

“The hardest challenges in testing are internal processes, especially the need to work collaboratively to define a problem. Most managers are good at asking questions, but not as good (or, for various reasons, reluctant) at specifying what would constitute a feasible answer to those questions.”

Read more

Balancing Network Effects, Learning Effects, and Power in Experiments

DoorDash Engineering Blog | By Wei Feng and Jared Bauman

“At DoorDash, we rely on experimentation to make decisions regarding model improvements and product changes because we cannot perfectly predict the results in advance. But experiments conducted by the Dasher team aimed at improving delivery quality and Dasher experience face challenges that arise from conflicting goals”

Read more

A/B Testing for Product Managers

Medium Blog @andrewhon_95925 | By Andrew Hon

“A/B testing (aka controlled experimentation) is used by top tech companies to build better product. I’ve built A/B testing at Disney Interactive and at Tinder, and the use of A/B testing continues to grow across the tech industry. This article is derived from a talk I gave to developing product managers at Product School in 2021, as a kind of practical introduction. Here’s why you should be A/B testing too!”

Read more

The Improvement of Our Randomization Algorithm

TowardsDataScience Blog | By Qike (Max) Li and Samir Jamkhande

“Recently at Wish, we discovered a sample ratio mismatch (SRM) in an A/A test. SRM refers to
the mismatch between the sample ratio set by the experimenter and the observed sample ratio. SRM indicates potential data quality problems that change the outcomes of your experiments . Microsoft shared an example in which the A/B test results were flipped after resolving the SRM. But, how is it possible that SRM presents in an A/A test?”

Read more

How A/B Testing Helps Microsoft and Why You Should Consider It Too

Medium Blog @rtkilian | By Rhys Kilian, Data Analytics @ Movember

“After reading this article you will understand the benefits of A/B testing, how Microsoft uses it, and some potential pitfalls.”

Read more

Experimentation on news websites: How we differ from other businesses

Medium.com NewsUK Technology | By Sabine Kejlova, Experimentation Analyst

“There are many factors at play of course, including industry type, objectives and team set-ups. For news organisations, there’s even more to consider. Here are three differences that I have found set experimentation at News UK apart from companies I have previously worked for.”

Read more

When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK?

Linkedin Newsletter | By Rommil Santiago

Rommil shares his perspective on this question and what you should consider when running two a/b tests at once.

Read more

5 Reasons Why Experimentation Programs Fail

Growth Hackers Medium Blog | By Jordana Rauber, Head of Marketing at Growth Hackers

“Building and running a successful experimentation program is no easy task, and there are many twists and turns to it that can keep you from reaching your goals. Here are a few of the most common mistakes we see in experimentation programs, and how to overcome them.”

Read more

Optimizely, Google Cloud forge new partnership

VentureBeat.com | By Tim Keary

“Under the new partnership, Optimizely will bring its Digital Experience Platform onto the Google Cloud, including its Web Experimentation and Full Stack Experimentation software.”

Read more

Who’s Hiring?

Digital Content Strategist| Heifer InternationalRemote, USA

Web Developer| Heifer InternationalRemote, USA

Web Producer| Heifer InternationalRemote, USA

Website Product Owner| Best EggWilmington, DE

Digital Optimization Strategist| WiderfunnelVancouver, BC

Digital Programs Manager| WiderfunnelVancouver, BC

Ecom Growth Optimization Manager| adidasPortland, OR

Sr. Marketing Manager, Experiments| AmazonSeattle, WA

Experimentation Lead, Web Presence| StripeNew York, NY

Lead, Experimentation| Macy’sNYC, NY

Senior Software Engineer| The Walt Disney CompanySeattle, WA

Digital Marketing Manager, Global Acquisition and Testing| Amazon Web ServicesSeattle, WA

Senior Software Engineer, Experimentation| Dow JonesNew York, NY

Subscribe to get experimentation insights straight to your inbox.