Welcome back to another edition of the Widerfunnel Experimentation News Roundup! These are the headlines that caught our attention over the last month, along with upcoming events and job postings you may be interested in. Did we miss something that should be included here? Let us know in the comments!
Experimentation News: November 2019
‘Digital is the testbed’: Why the 2020 election is focused on online advertising – DIGIDAY.com – By Kristina Monllos
Algorithmically, candidates benefit from experimentation. “The more messages you put out the better your campaign does,” said Matthew Rednor, founder and CEO of Decoded Advertising. “We’re seeing that for clients there’s a tipping point where actually once you’re over five messages, once you’re in the 50-to-60 range, your CPAs start to drop and your ads start to perform better. Also, TV is not an agile enough channel to allow you to put out that many messages.”
A/B Testing — What Role Does it Play in the Era of Machine Learning? – Capital One Tech – by Vito Covalucci
“If we’ve limited our changes to as few variables as possible, we can learn what actually causes changes in behavior. The ‘why’ is more challenging, but the ‘what’ becomes clear. In the complex, multivariate world of machine learning, finding causes is not the primary concern. Optimizing an objective function is. Therefore, for humans to learn and to create new ideas and build models that reflect the ideal world, A/B testing fills a valuable and lasting role.”
Breaking Down the Barriers to Innovation – Harvard Business Review – By Scott D. Anthony, Paul Cobban, Rahul Nair, and Natalie Painchaud
In our work and research, we’ve found that the most innovative organizations exhibit five key behaviors: They always assume there’s a better way to do things. They focus on deeply understanding customers’ stated and unstated needs and desires. They collaborate across and beyond the organization, actively cross-pollinating. They recognize that success requires experimentation, rapid iteration, and frequent failure. Last, they empower people to take considered risks, voice dissenting opinions, and seek needed resources.
“Above all else, I base my decisions on the development leader’s way of thinking,” says Furukawa in a rare interview with Western media, and who spoke with TIME mostly through a translator provided by the company. “Nintendo is Nintendo because of our games, characters and IP. So giving our teams the freedom to experiment with new ideas is something I strongly agree with. Expansion can’t happen without the freedom to try something new, and the courage to step into unfamiliar territory.”
So why do some nudges work while others do not? Quite simply, many of these failed nudges aren’t nudges at all; they’re nags. And no one likes being nagged.
A nudge, as opposed to a nag, leverages cognitive heuristics to prompt slight behavior changes without people consciously reacting to the nudge. In other words, a nudge isn’t something you realize you’re getting, and the result isn’t something you think too much about. A nudge happens in the background of your daily life, and it works best when the goal is to slightly increase a positive outcome.
Putting experimentation at the core of your business can increase the efficiency of your advertising campaigns and help you make smarter decisions. However, realizing the full potential of measurement often entails making deep organizational changes. [Direct WhitePaper Link]
Some interesting experiments people are noticing:
- Microsoft experimenting with new “surprise me” feature
- Instagram experimenting with hiding ‘likes’ for some users
- Twitter is experimenting with new features to fix the ratio
- LinkedIn Is Experimenting with New Handy Feature for the Ease of Its Users
- Apple experimenting with ways to automatically create Memoji characters based on selfies
- GolfTV is experimenting with Netflix-style ‘Originals’ to fuel streaming app
#TBT: The Anti-Poverty Experiment – Wall Street Journal – By Jason Zweig (Note: This is from June 2015)
“The daunting realization is that we don’t know what the hell we’re doing in most fields of life, especially the ones that involve people,” says Richard Thaler, professor of behavioral science and economics at the University of Chicago Booth School of Business and author of the new book “Misbehaving: The Making of Behavioral Economics.” He adds, “The alternative to guessing is to run experiments.”
Upcoming Experimentation Events and Webinars
- [Webinar] Driving Business Growth in the Experimentation Era – Nov 21 2019 – Hosted by Widerfunnel
- [Webinar] The 3 product experimentation pitfalls & how to avoid them – Dec 11 2019 – Hosted by Optimizely
Past Events and Webinars from Widerfunnel
- [Webinar] How to Optimize Your E-Commerce Website at Every Customer Touch Point – Aug 15 2019 – Hosted by FullStory/Widerfunnel
- [Video] How Disruptors Experiment Faster and Smarter (and what you can learn from them) – Opticon19 Session – Hosted By Widerfunnel/HP
- Shutterfly, Inc. – Sr. Manager, A/B Testing and Optimization (Redwood, CA)
- Lowe’s Companies, Inc. -Senior Manager, Experimentation Program – A/B Testing (Charlotte, NC)
- BrainWorks – Director of Digital Analytics & Experimentation (Seattle, WA)
- Lyft – Product Manager, Experimentation (San Francisco, CA)
- Convoy – Head of Product, Shipment Growth (Seattle, WA)
- The Mice Groups, Inc. – Product Manager – Experimentation/Adobe Target (Pleasanton, CA)
- Amazon – Sr. Applied Scientist – Weblab Team – A/B Test Experimentation (Seattle, WA)
- Thomson Reuters – Sr. Digital Testing and Optimization Manager (Carrollton, TX)
- Agoda – Head of Experimentation (Seattle,WA)
- Avis Budget – Director, Insights & Analytics (New Jersey)
- Credit Karma Senior Growth Marketing Manager (San Francisco, CA)
- Hulu – Senior Software Developer – Experimentation (Seattle, WA)
- Hulu – Experimentation Manager (Santa Monica, CA)
Do you know of an upcoming event or current job posting we should include in this round-up? Let us know in the comments section below, and stay tuned for our next experimentation news round-up next month!
Subscribe to get experimentation insights straight to your inbox.