Experimentation news round-up #5: Testing tool recommendations, Spotify’s viewpoint, and jobs galore!

By: Chris Goward
11 min. read Last updated: April 24th, 2020

Welcome back to another edition of the Widerfunnel Experimentation News Roundup! These are the headlines that caught our attention over the last two weeks, along with upcoming events and job postings you may be interested in. Did we miss something major? Let us know in the comments!

experimentation news round-up articles

Experimentation News: August 2019

Spotify Embraces Tech Roots To Fuel Authentic Growth In Streaming Landscape – Forbes

While many equate Spotify to music, the foundation of the business is technology. The concepts of “experimentation” and “iteration” — both central to product development of a tech company — were repeatedly mentioned on a recent investor call, pointing to Spotify’s roots. “We’re actively prototyping new products with our partners,” said CFA Paul Vogel with respect to experimentation and in response to a question on why the Upload Beta Program was cancelled.

Does Your Organization Need a Director of Personalization? – CMSWire.com

According to Andy Zimmerman, CMO at Evergage, the director of personalization would ask questions like: Who knows these products? How do we adjust our algorithms? How do we test new ones? How do we run A/B testing? “For answers, that person would need to pull in other roles,” Zimmerman said. “For example, a designer, a data analyst, a developer and others, depending on the channel: email marketers for email personalization, etc. Coordinating across these teams, the director of personalization would direct the activities to execute against the company strategies.”

Kerger commits to continue leading PBS through ‘time of great experimentation’ – Current.Org

“We’re at a time of great experimentation,” said Paula Kerger, PBS’s longest tenured president,“In terms of the platforms and the formats — this is not a time for the faint of heart. If you feel like you need everything mapped out, you definitely shouldn’t be in media, and you certainly shouldn’t be in public media.”

An immediate revenue boost: Why you should be prioritising digital experimentation – SmartCompany.com.au

Velocity is key. According to leading A/B testing and personalisation vendor Optimizely, organisations running between one and 20 experiments per month are, on average, driving 1-4% increases in revenue; whereas organisations that are running 21 experiments or more, are most likely to drive a revenue increase of 14% or more.

In addition, according to Optimizely and VWO, the average ‘win rate’ for A/B tests is between 7% and 10%. Therefore, typically, for every 10 tests that a company runs, only one of those will be a statistically significant winner.”

8 of the Best A/B Testing Tools for 2019 – HubSpot.com

“Every company has a different set of customers, so there’s no one-size-fits-all formula for designing the most optimal website, crafting the most compelling copy, or building the most effective product. To figure out which website design, line of copy, or product feature will produce the best results for your company, you must discover what your unique set of customers prefer.”

Why A/B testing is crucial for effective digital communication – Medium (Sina Oerder)

“A/B testing is commonly known for website content. But besides this, A/B testing is crucial for all kinds of digital communication. By applying A/B testing in social media, for example, you are able to influence the success of your campaigns by steering different variables. Utilizing the settings of different social media platforms, it is possible to test nearly everything that is of interest to you. In this way you can find out what it is that makes your campaigns most successful.”

Five key tips to drive value from content and martech experimentation – MarketingTech.net

“As Jeff Bezos, CEO of Amazon, said in the latest letter to shareholders, ‘as a company grows, everything needs to scale, including the size of your failed experiments’.

Uber and Airbnb are two examples of organisations that have been built on a foundation of experimentation and adopt a customer-obsessed mindset. This means they can thrive in today’s highly competitive digital landscape, and turn user data into a competitive advantage by pairing high-volume experimentation with quality performance.”

The Merits of Storytelling in Digital Marketing (When to Use A/B Testing and Storytelling) – AB Tasty

“Marketers must clearly identify the three core elements in their message before spinning their tale: the introduction, the narrative arc, and the resolution. Take a step further, and adopt the classic storytelling technique which has stood the test of time: the Hero’s journey.”

How today’s news publishers can use data, best practices, and test-and-learn tactics to build better pay-meters – Harvard Kennedy School, Shorenstein Center On Media, Politics and Public Policy

“Our research suggests publishers should invest in capabilities to engage in constant testing and experimentation in digital — to build engagement among digital audiences and ultimately convert engaged readers into paying subscribers. For commercial and for-profit models in particular, publishers should become smart across a range of new strategies, including how to configure meters and rules to calibrate a mix of free and paid access.”

Learning from the Digital Leaders – Bain & Company

“Digital leaders are fast, particularly at decision making and execution, enabling them to experiment and pivot when necessary. They identify and organize around the technology trends that matter most. And they’re adept at orchestration, moving effectively from experimentation to full-scale transformation.”

How L’Oréal’s test-and-learn mindset has kept it on top of Hong Kong’s beauty market – ThinkWithGoogle.com

“In the past few years, new digital tools and ad formats have unlocked our ability to engage consumers with tailored, timely messages. Our campaigns have gotten more agile and effective, and we’re removing friction between our online presence and brick-and-mortar stores. And it all started with the strong spirit of experimentation and test-and-learn mindset that are deeply ingrained in L’Oréal’s DNA.”

experimentation news roundup - upcoming events

Upcoming experimentation events and webinars

Learn how to run high-impact experiments from Product/Marketing Leaders in Toronto – Aug 29th

Once a quarter, CareGuide brings in the most innovative Product and Marketing leaders in Toronto to their office to discuss their TO community’s learning from running high-impact experiments. Through their panel discussions and talks given by experts, you’ll learn:

  • What constitutes a culture of growth and optimization
  • Tips for building that culture in your own company
  • Lessons from product and marketing leaders who embrace the test-and-learn approach

Opticon 2019 North America – San Francisco – Sept 11-13

One of the largest experimentation events in North America. If you are in the experimentation space in United States or Canada, you will want to attend Opticon 2019 in San Francisco this fall.

Opticon Europe 2019: London – Oct 1st

Optimizely’s European conference, hosted out of London and geared towards Europe-based companies.

Prioritizing Fast Research for Agency Timelines – Aug 27th – Online

Want to learn more about incorporating qualitative research into your work streams? Join agency and consultancy research pros, including Widerfunnel’s Director of Behavorial Sciences, Matt Wright, to learn how the research world is keeping pace with customer needs.

experimentation news round-up articles

Current job postings in the experimentation space

Do you know of an upcoming event or current job posting we should include in this round-up? Let us know in the comments section below, and stay tuned for our next experimentation news round-up in two weeks!

some image

Discover how your experimentation program stacks up!

Benchmark your experimentation maturity with our new 7-minute maturity assessment and get proven strategies to develop an insight-driving growth machine.

Get Started