October 2020: Experimentation News Round Up

By: Daniel Molina
25 min. read Last updated: November 20th, 2020

logoExperimentation News

A Product Manager’s Guide to Experimentation

Medium.com Blog | By Alex Mitchell

Product Managers develop hypotheses on an almost daily basis. It’s a critical part of the job.

Read more

Episode 4: Airbnb, Ronny Kohavi, VP and Technical Fellow

Widerfunnel Blog | Hosted by Chris Goward

In this episode, we have a great conversation lined up! My guest is Ronny Kohavi, Vice President and Technical Fellow for Relevance and Personalization at Airbnb.

Read more

Creating experimentation as a habit

Medium.com Blog: Bootcamp | By Bithika Mehra

Building an experimentation culture at your organization is similar to creating and sticking to a good habit. Learn how to apply those lessons.

Read more


AlexBirkett.com | By Alex Birkett

A/B testing is, at this point, widespread and common practice. But even if we all agree on the value of experimentation, there’s a ton of debate and open questions as to how to run A/B tests.

Read more

Jeff Bezos: Amazon turned into ‘the everything store’ thanks to an email to 1,000 random people

CNBC.com | By Tom Popomaronis

Those who don’t embrace failure, Bezos said, will “eventually get in a desperate position where the only thing they can do is a Hail Mary bet at the very end.”

Read more

The World Needs More Intrapreneurs to Drive Change

Forbes | By Chris Goward<

Wherever there’s a positive change in an organization, you will find a champion leading. I believe 2020 will be remembered as the year of accelerated change.

Read more

Partnering with HiPPOs and leveraging neuromarketing to drive conversion

Experiment Nation | By Rommil Santiago

Despite our best efforts, HiPPOs are key stakeholders that can make or break an optimization program. I recently spoke to Bithika about her thoughts on how to get HiPPOs on our side.

Read more

Improving Online Experiment Capacity by 4X with Parallelization and Increased Sensitivity

DoorDash Engineering Blog | By Jessica Zhangg, Yixin Tang

When testing consumer-facing features, we encountered two major challenges that constrained the number of experiments we were able to run per quarter, slowing the development and release of new product features.

Read more

How COVID-19 has pushed companies over the technology tipping point—and transformed business forever

McKinsey.com | Edited by Daniella Seiler

[Successful companies] are also more likely than others to report speeding up the time it takes for leaders to receive critical business information and reallocating resources to fund new initiatives. Both are key aspects of a culture of experimentation.

Read more

3 ways retail stores can survive — and maybe even grow — against the odds

MIT Management Sloan School | By Sara Brown

How can retail stores survive online disruption and a pandemic? Start with focusing on the customer experience and creating good loyalty programs.

Read more

Digital now: Traditional airlines not ready for post-COVID travel

Digital Journal | By Tim Sandle

45 percent of airlines said they do not have a budget for digital optimization and experimentation and 49 percent of airlines do not run experiments at all or only do them on an ad-hoc, project basis.

Read more

Chipotle: ‘Nothing Is Off The Table’ When It Comes To Innovation As Digital Sales Jump By 202%

Forbes | By Alicia Kelso

COVID caused trial and experimentation and if you give people an easy experience, like we do, they tend to come back and we’ve seen that.

Read more

Experimentation is a Marketer’s Secret Weapon. Here’s Why

Business2Community | By Pini Yakuel

The majority of marketers who prioritize testing and experimentation significantly outperform their peers.

Read more

Experimentation in an age of risk

ITProPortal | By Jill Maassen

Brands can use a ‘test and learn’ approach as a guiding principle to secure their position in today’s reimagined world of business.

Read more

A top MoneyGram exec lays out a digital transformation that’s helped fuel triple-digit growth in cross-border payments during the pandemic

Business Insider [Paywall] | By Shannen Balogh

“One of the first things that I did when we embarked on this digital transformation was to ensure that we were sticking to our principles of experimentation” she added. “Sometimes that has to be done by edict.”

Read more

Make room for intelligent failure, Deloitte exec says

CIODive | By Roberto Torres

The shifts brought about by the digital age calls for experimentation, but employees won’t feel enabled to experiment on the job unless they feel safe to fail.

Read more

How The Trade Desk’s Crystal Eastman Fosters Curiosity and ‘Joyful Experimentation’

AdWeek | By Luz Corona

It’s important to me that our team stays culturally curious to their surroundings–in and out of work–and provide room to test out new ideas. I try to keep the process of experimentation joyful; celebrating epic losses and much as big wins to make sure the team feels safe to take risks.

Read more

4 Steps To Turn Makeshift Digital Transformation Into Long-Term Success

Forbes | By Tomoko Yokoi

We recommend that companies strive for a test-and-learn environment that supports hypothesis-driven experimentation with a rapid turnaround.

Read more

Preparing For The Unknown: Black Friday 2020

Forbes | By Aater Suleman

Retailers must employ a strategy that is differentiated, supported by an agile technology platform that allows them to easily experiment, fail cheap, and measure results accurately.

Read more

Digital tools have empowered staff during the pandemic, says M&S stores boss

RetailWeek | By Rackspace Technology

“Before people were afraid to ask but now there’s a spirit of openness and asking ‘what if?’. There are more experimental projects going on that retailers can do in a few weeks and a few thousand pounds into rather than investing in multimillion-pound projects.”

Read more

Embracing experimentation: How to prove media impact in the new normal

Think With Google | By Karen Stocks

One of the best ways to know if your marketing is working is by running experiments. These experiments offer marketers a clearer understanding of what’s performing well and what can be improved.

Read more

The Three Biggest Digital Marketing Mistakes of 2020

The Financial Brand | By Kathryn Turnoff

The Vision 2021 report shows that organizations admit that they still have a way to go on these low-cost-of-entry tools: only 34% are currently using A/B testing, and a mere 12% are employing personalized web pages.

Read more

Optimizing Your Site For Conversions: A Primer On WordPress A/B Testing

VWO Blog | By Shubhi Ahluwalia

In this post, we discuss how you can leverage A/B testing on your WordPress site and how it could be a game-changer for your business.

Read more

logoEvents and Webinars

Past webinars / events:

Culture of experimentation: How organizational culture can make or break your testing program

Beth Foster, the leader of the Microsoft Digital Stores Experimentation team, shares the approaches she has taken to build a culture of experimentation, as well as her successes and failures along the way.

watch the recording

Enterprise Innovation: How HP Instant Ink Successfully Launched a Disruptor Brand

Find out how you can create an agile, digital-first, customer-obsessed program that drives both product innovation and optimization.

watch the recording

logoWho’s Hiring

Senior Product Manager, Growth| Change.orgSan Francisco, CA

Intuit| Principal Product Manager, Intuit Experimentation Platform (IXP)San Diego, CA

Senior Product Manager – Growth| AcornsIrvine, CA

Senior Manager, A/B Testing| Realtor.comSanta Clara, CA

VP of Product Innovation, Growth| NetflixLos Gatos, CA

Product Manager, Experimentation and Release| WaymoMountain View, CA

Engineering Manager, Experimentation| UberSan Francisco, CA

Product Manager – Experimentation| LyftSan Francisco, CA}

Product Manager, Growth| PinterestSan Francisco, CA

A/B Testing – Product Manager| Carter’sAtlanta, GA

Senior Support Planner for Experimentation| MicrosoftIrving, TX

Lead, Digital A B Testing| Frontier CommunicationsIrving, TX

Director, Growth & Experimentation| Campaign MonitorRemote, US

Experimentation Strategist| WiderfunnelVancouver, BC

Director of Marketing, Conversion| ZazzleFort Collins, CO

Product Manager| Epic GamesCary, NC

Manager, Data Science Experimentation| Disney Streaming ServicesNYC, NY

Product Manager (E-Comm)| HBO MaxNYC, NY

Product Lead, Subscriber Growth Platforms| New York TimesNYC, NY

VP, Growth Marketing| MongoDBNYC, NY

Director of Experimentation, Growth| GympassNew York City, NY

some image

Discover how your experimentation program stacks up!

Benchmark your experimentation maturity with our new 7-minute maturity assessment and get proven strategies to develop an insight-driving growth machine.

Get Started