Case Study Executive Summary
Industry: Software; Technology
Business Model: Lead generation
Experimentation Champion: Michele Pendergast, Head of Demand Generation
- Short-term: Increase lead generation from newly redesigned website
- Short-term: Increase overall conversion rate among visitors from paid campaigns
- Long-term: Drive iterative, sustainable growth for the company
- Long-term: Build a culture of experimentation at Envoy
- Source WiderFunnel as an experimentation partner to provide strategic support and staff augmentation
- Quick launch of evidence-backed A/B tests to generate lift and company-wide buy-in
- Continued internal education around conversion optimization and experimentation
- Back-to-back winning experiments, resulting in double digit conversion rate lift
- Increased buy-in for conversion optimization at the Executive level
- Opportunities for experimentation program expansion deeper into the sign-up funnel and product
I knew that I could count on [WiderFunnel] to not only deliver on experiments, but to also help us build out the overarching program the right way from the beginning.
The full story
Envoy was born in Silicon Valley. The brainchild of Larry Gadea, a former Twitter and Google engineer, the Envoy app offers an alternative to the analog systems companies use every time they welcome visitors.
In his career, Gadea had seen the custom-built sign-in systems used by larger tech companies and wanted to build a solution for all companies. A solution that would enable a smooth, professional experience when welcoming guests.
Over the past 6 years, Envoy has seen major growth. In 2015, the company received $15M from Andreessen Horowitz in Series A funding. By 2017, over 5,000 companies were using Envoy’s services and the company had developed a new product to manage office deliveries.
In 2018, five years after its launch, Envoy was being used in over 10,000 workplaces, with four of five users outside of the Bay Area. Last Fall, the company raised another $43M in Series B funding to pursue the vision of “the smart office”.
While Envoy continues to enjoy rapid success, leaders are focused on laying the foundation for long-term sustainable growth. Part of this foundation is an experimentation machine that generates continuous growth and insights.
Get a 1-page Executive Summary of this case study
Enter the Experimentation Champion
When Michele Pendergast joined Envoy as Head of Demand Generation, she came in with a plan. Formerly a Marketing Manager at Intercom, Michele had learned first-hand the importance of applying rigor to conversion rate experiments on a marketing website.
Envoy had reached a growth plateau. To this point, Gadea had been working to generate buzz in the media, but things were slowing. The company hadn’t had anyone in a Marketing role for over a year, and was struggling to drive traffic to their website and organize data to understand visitor behavior.
Michele’s priorities were clear:
- Manage the money that was already in flight. Envoy had ads in play already and Michele wanted to ensure these ads were being managed with skill—so she hired a paid media agency.
- Improve organic traffic to Envoy’s marketing website. Michele brought in a SEO agency to improve website infrastructure.
Given that both of these activities are meant to drive traffic to the Envoy marketing website, Michele’s third priority was clear: To ensure that the site was actually converting people.
If I’m spending money to drive more people to the site, I want to make sure when they land on the site, that the site is set up to convert, to capitalize on that increased traffic. It was a very obvious domino effect.— Michele Pendergast
Michele knew that the most effective way to convert users at a higher rate was to implement a rigorous optimization program based on experimentation. And as with her other priorities, Michele knew she wanted to bring in an agency partner to help.
“Since we were such a small team, I knew that we needed an agency partner that felt like a true extension of our team. We weren’t just looking for consultants that would provide a plethora of options; we needed a team of experts that were as invested in our success as we were. Having worked with WiderFunnel before, I knew that I could count on them to not only deliver on experiments, but to also help us build out the overarching program the right way from the beginning,” explains Michele.
Introducing “conversion rate optimization” at Envoy
But before she could move forward, Michele needed to get her boss, Russ Thau, the company’s VP of Revenue and Success, on board with her plan. While Russ has an allstar sales background, he wasn’t super familiar with the concept of “conversion rate optimization”. This is where Michele began her efforts around education.
What is conversion rate optimization?
At WiderFunnel, we define Conversion Optimization (or Conversion Rate Optimization) as the science and art of getting a higher percentage of your digital visitors to take action and become a lead or customer.
Science: you cannot simply “guess and hope” which changes to your digital experiences will achieve a higher conversion rate. Instead, you need to develop valid hypotheses, run controlled experiments, and evaluate results to lock in the improvement.
Art: a statistician or engineer alone cannot create the visuals and messaging that engage your digital visitors. You should leverage a structured process that empowers your team to creatively apply learning from thousands of test results, to test only the best variations on your website.
Low literacy around CRO was one factor, but Michele needed to handle the optimization conversation with care for another reason. Envoy had just finished a website redesign and the team was fully and heavily invested in the success of the new site. She anticipated pushback if she couldn’t properly articulate her intentions with experimentation.
Michele worked carefully to explain that, while the redesign had certainly succeeded from a brand and aesthetic angle, there were still points of friction in the user experience. She emphasized the missed opportunity of not addressing these potential barriers to conversion.
She also made it clear to Russ that experimentation would be a cross-functional effort. This wasn’t about performance marketing making a play on its own. Rather, it would be a collaborative effort to iterate on the overall customer experience.
Getting the rest of the Envoy team on board
After getting the green light from her Executive sponsor, Michele wanted to address potential concerns on Envoy’s Design and Engineering teams. Design wanted to ensure that they were still heavily involved in any visual design decisions and changes made to the site. And the Engineering team was concerned about the impact running tests would have on website performance.
Michele knew that the best way to get Design and Engineering on board was to include them early in the process and be as transparent as possible about what experimentation would mean for them. First, she connected Envoy’s Senior Software Engineer, Alex Hollenbeck, with WiderFunnel’s Lead Engineer, Tony Ta.
“I told him to ask any questions. They spoke for a while about WiderFunnel’s processes and how their technical team would acclimate to our dev environment. And afterward, Alex turned to me and shared his enthusiasm for the program. He was excited to get started,” recounts Michele.
Since Envoy kicked off their experimentation program, the experimentation team has worked to include Envoy’s Designers wherever possible, and as early in the experiment development process as possible.
I try to set expectations that we’re all going to have different opinions. That we’re all coming at this from our different areas of expertise—which is great. That said, we should be able to have open and honest conversations about our ideas and determine, as a team, which ideas are best for the company and the goals we’re trying to achieve.— Michele Pendergast
The first experiment
Because experimentation education was needed, Michele and her counterparts at Widerfunnel decided to push for “simpler” tests at the outset. This thought process reflects WiderFunnel’s prioritization framework: PIE.
PIE is a framework that enables organizations to prioritize where (and what) to test first based on three factors: Potential, Importance, and Ease. Potential is how much improvement can actually be made in a given area. Importance is how valuable a page or funnel is (e.g. how much paid traffic does it receive; how visible is it, etc.). And finally, ease is how easy it is to test in a certain area, both technically and politically.
Given the level of understanding around “CRO” at Envoy, political Ease played a role when Michele and her WiderFunnel team were determining where to run Envoy’s first experiment. The team identified the site-wide navigation as an important, high-impact area for testing, where we could run a simple experiment that required minimal engineering work and easily fit within Design standards.
We came in knowing that the site had just undergone a full redesign so we knew that certain areas would be politically sensitive. From a data perspective, the site-wide navigation made perfect sense to kick things off. It receives the highest amount of traffic on the site and has high potential for driving results.
We wanted to start with the nav in order to prove the worth of experimentation, while encountering the least amount of pushback from the different teams involved.
The experiment details—site-wide navigation
As soon as Envoy signed on with WiderFunnel, the team began the Explore phase. Explore is the first phase in our Infinity Optimization Process, wherein we gather as much information as possible about an organization and its customers. Explore insights come from customer research, analytics, stakeholder interviews, persuasion principles, business objectives, the experiment archive, and more.
Explore is a continuous process. The insights gathered inform every experiment we run, and insights that are uncovered when an experiment concludes are fed back into the Explore phase—hence the infinity loop.
During Explore with Envoy, Strategist Neil Lim collected user research data and digital analytics data. The team also conducted several LIFT analyses, all of which revealed several potential friction points in the site-wide navigation.
- CLARITY: There is no easy way to see the features for both Envoy’s Visitor and Delivery products or what distinguishes them from each other.
- CLARITY: It isn’t clear that Envoy Visitors and Envoy Deliveries are two exclusive products.
- CLARITY: Do the value proposition points in the “Why Envoy” dropdown apply to both products or only one?
- ANXIETY/DISTRACTION: The value proposition points in the dropdown are easily missed if no one hovers over the “Why Envoy” menu.
After much discussion around these points, the team proposed several hypotheses. Ultimately, we decided to focus on a small change in the first variation: Renaming the “Why Envoy” dropdown to “Solutions”. According to Envoy’s user research, the term “solutions”—specifically “visitor management solutions”—was important to Envoy’s visitors. It was a term they were using.
With this variation, we wanted to make it clear that the value proposition points hidden in the “Why Envoy” drop down were actually solutions to visitors’ queries.
Hypothesis: Changing the “Why Envoy” label to “Solutions” will communicate that the key value proposition points found within this dropdown are the benefits users can derive from using Envoy’s products, increasing their motivation to sign up for a trial, resulting in more sign ups.
With variation B, we wanted to explore a slightly different angle. Signing up for an Envoy account is free and easy, a potential value proposition point that was only mentioned towards the bottom of the page—we wanted to see if emphasizing this value point in the navigation would increase sign ups.
This experiment idea was also bolstered by a past experiment we’d run with a SaaS client, which resulted in an almost 9% uplift in signups:
As a clarity point, we also wanted to test spacing out the links in the navigation. The overall hypothesis for variation B was:
Leveraging key value proposition points around the call-to-action and reducing the number of actions users need to take to complete signing up for an account will reinforce the fact that Envoy is risk-free, resulting in more visitors signing up.
In this variation, we shifted the primary navigation links to the left and added an email capture field into the nav.
Ultimately, variation A resulted in +16.53% increase in signups at 99% statistical confidence—a huge first win for Envoy. Variation B had a flat result, which goes to show that best practices mean nothing if they aren’t validated within your specific context.
In variation A, the team observed greater engagement with all key navigation links and reduced engagement with the “Sign Up” call-to-action. Visitors were exploring other pages on the site to a much higher degree, particularly the Visitor Management product page—which is Envoy’s most valuable product. While fewer visitors clicked on the CTA within the nav, sign ups increased drastically.
This indicates that Envoy visitors may need more education around the platform itself before they are willing to sign up for an account. Based on these results, future experiments should focus on either educating visitors about the benefits of the platform or making it easier for visitors to access this type of content.
Socializing the first experiment win
When WiderFunnel Strategist, Neil Lim, presented this result to the Envoy team, they were pumped. So pumped, in fact, that Alex hardcoded the winning nav experience right there on the call.
Then Michele ran (literally) to tell Russ. She explained the experiment and the result verbally—16.5% uplift!—and promised to follow-up with a more comprehensive results analysis, provided by her counterparts at WiderFunnel.
As promised, Michele packaged the results analysis to share in writing with Russ. Russ then took those results and socialized them to Envoy’s CEO as well as the rest of the Executive team. It was important to Michele that Russ socialize this first experiment to really highlight the team’s success, reinforcing the excitement and enthusiasm around the experimentation program in general.
This was the perfect first win. We were able to show how even small optimizations could lead to significant returns. It set us up to start feeling more confident and excited about the bigger opportunities ahead of us.— Michele Pendergast
The next experiment
Off the heels of this first win, Michele could feel the momentum. She was starting to get that much sought after organizational buy-in. For the next experiment, the team decided to tackle the homepage proper. But given the visibility and importance of this page, Michele wanted to make sure to do things the right way—the collaborative way.
Recommended next reading
How to blast through silo mentality to create a culture of experimentation
So, she brought Envoy’s Lead Designer, Amy Devereux, into the conversation early to ensure that the experiment variations would 1) make sense visually, and 2) align with her vision for the website.
The experiment details—the homepage
In conversations with the Envoy team about the homepage, one question that surfaced was around credibility: Namely, would highlighting Envoy’s customer logos impact sign-ups? These logos were potentially a major value proposition point—Envoy’s customers include large credible companies. However, they weren’t getting a lot of exposure in their original position on the homepage.
WiderFunnel strategists also considered evidence from psychological principles that would support moving the logos higher on the page. Namely the Bandwagon Effect, which states that we tend to put our trust in something because we believe many other people have put their trust in it first. There is safety in numbers.
So, the following hypothesis was put forward: Increasing the prominence of brands that use Envoy will alleviate any anxiety associated with signing up for an account, while also enhancing the credibility of Envoy which will ultimately result in increasing trial sign-ups.
Alternatively, with variation B, the team wanted to test emphasizing Envoy use cases. This idea came out of the initial LIFT analysis. From a value proposition perspective, the information contained in the use cases section is useful and beneficial, but may have been missed because it is in between two large content blocks.
There was also click data from FullStory which indicated the use cases section was not receiving any clicks, re-validating the lack of exposure of this section. The WiderFunnel team also wanted to use this variation to funnel Envoy’s visitors to a single product: Visitor Management. This reflected an overall business objective.
For variation B, the hypothesis was:
Communicating the benefits of using Envoy and directing visitors to a single primary product will encourage visitors to learn more about the platform, ultimately resulting in increasing sign ups.
Both variations were winners, with variation A resulting in a 12% lift in sign ups and variation B resulting in an 8% lift in sign ups.
Given that most visitors may be unfamiliar with modern workplace technologies like Envoy, exposing them to prominent customers may have resulted in:
- Spurring their curiosity about the products and driving them learn more on key Envoy pages
- Helping to alleviate anxiety involved in signing up for a trial after seeing prominent companies using the platform (Bandwagon Effect)
The future of experimentation at Envoy
With two strong experiment results under her belt, Michele sees blue skies ahead for experimentation at Envoy. From her point of view, there are two main priorities to keep momentum going:
- Continue to socialize experimentation at the Executive level, and
- Level-up Envoy’s approach to rigorous experimentation
Sharing these initial experiment results has already led to increased organizational buy-in, and opened new doors for the experimentation program. The team is hoping to expand to program to test on the sign up funnel as well as other areas and products for Envoy.
“I’ll probably continue to share every experiment result with my VP as soon as I get it—I get really excited and want him to be excited, too,” laughs Michele. “But moving forward, I will want to be more strategic about how I bundle and share results within the organization at-large.”
Michele is building a communications plan that will allow her to spread the word as effectively as possible. She has started providing in-depth updates (hypotheses, experiment designs + rationale, results) in weekly Marketing Team meetings and rolling up high-level updates (experiment hypotheses + results) post-experiment into Envoy’s weekly leadership meetings.
As for the overall experimentation strategy, the team wants to focus on getting into more strategic and higher impact experiments. Where the first few experiments were focused at addressing high-impact, low-hanging fruit, subsequent experiments will focus on improving customer lifetime value and trial conversion rates for Envoy.
These are the metrics that will really drive sustainable growth and revenue for the company.
Everyone at Envoy is here to help elevate the workplace experience. We have a shared vision for the future. No matter your department, we’re all rowing in the same direction. WiderFunnel helped us kick off an inclusive growth experimentation program so we could play to all of our strengths, getting us a little closer to that future, one test at a time.— Michele Pendergast
Add your voice to this year's "State of Experimentation" survey!
We just launched our annual "State of Experimentation" survey. If you're involved in digital experimentation, we would really appreciate your input. All respondents will receive the complete, global results ($5K value) for free. Survey closes June 23rd!Take the Survey >