Welcome to our Google Optimize tutorial!
Today, we’ll be discussing how to A/B test a website with the free experimentation tool from Google.
After building a sound conceptual foundation, we will familiarize ourselves with Optimize’s most popular features before diving deep into the specifics of how to set up, run and analyze an experiment.
This is more than an introduction to the world of Google Optimize as we’ll throw in pro tips to help you get the most out of it.
This is a practical read but not a short one. If you prefer to jump into action then I strongly recommend at least learning what we detailed in when you should optimize.
What we’ll cover in more detail:
- What is Google Optimize?
- What Is A/B Testing in Google Optimize?
- Why Should I Use Google Optimize?
- When Should I Use Google Optimize?
- How to install Google Optimize? (with and without Google Tag Manager)
- Creating an Experiment in Google Optimize (A/B Test)
- Targeting and Variants
- Measurement and Objectives
- Starting Your Test
- How Do I Analyze Google Optimize Results?
- How To Use Google Analytics and Optimize
Let’s get rolling.
What Is Google Optimize?
Google Optimize is a free tool from Google that allows you to improve the experiences of users on your website. The platform allows you to present to your audience different versions of specific pages of your website and test the most effective one.
Google Optimize uses three different types of tests: A/B Testing, Multivariate Testing and Redirect tests.
For enterprise-level needs, there is a premium version called Google Optimize 360.
What Is A/B Testing in Google Optimize?
A/B testing is when visitors are shown two (or more) versions of a page to identify which one brings in more of the results you are looking for.
The main page is called the original (or control) and the modified version of that page is the variant. When you A/B test, you show both pages to different groups of people (randomly chosen) to see which one performs better.
The purpose of this test is to solve a user pain point or a problem such as a decrease in traffic or revenue. In general, these changes are not very big. They typically are limited to testing elements. This can be as simple as changing a button color or a headline, etc.
Why Should I Use Google Optimize?
We’ll cover the three most common reasons to use Google Optimize – integration, cost, and ease of usage.
First, Optimize is well-integrated. That’s because it’s one of the solutions of the Google Marketing Platform, which provides a suite of tools like Google Analytics for businesses all in one spot.
This means that Google Optimize can be linked to Google Analytics, Google Tag Manager, and Google Ads. It removes the need to switch between scattered platforms to get the marketing job done.
Follow these links to link your Google Analytics and Google Ads accounts to Google Optimize:
Second, it is free to use. Although the upgraded version (Google Optimize 360) provides additional capabilities, the free version is more than sufficient for most companies.
When GA4 replaced Universal Analytics, the change came with integrations, customizations, and additional features which used to be only available to Optimize 360 users.
Although Optimize 360 and other platforms may be more suitable for large corporations, the free version still competes well above average compared to many other paid testing solutions.
Third, it is easy to use. You don’t need to know how to code and therefore don’t need to rely on a developer for your tweaks.
If you know how to code or have access to a developer, then you’ll be able to design advanced experiments.
When Should I Use Google Optimize?
There are plenty of helpful tutorials online to get you up and running, as you don’t need much to create your first experience.
What this leaves out, though, are other prerequisites that go beyond installing the right Optimize snippet to your website. You’ll quickly realize that not everyone is ready for split testing.
The main concern relates to low-traffic websites, which refer to any site with less than 4,000 visitors per month.
The prerequisites I’ll mention here are professional advice. It should not discourage you from implementing A/B testing to learn or to improve your business. It’s just that your results will most likely be insignificant.
However, we’ll show you a workaround if your traffic is low.
Marketing Forecasts
What are your funnels and how close are you to your numbers? Can you confidently tell the conversion rates for each of your funnel steps for the next 1-3 months?
These questions are relevant because measuring behaviors indicates quantitatively where problems and opportunities lie. Not only that, but they also allow you to guide or come up with a hypothesis for your experiments.
Speaking of funnels, they don’t have to be complicated.
A simple 3 step funnel including micro goals and macro goals can go a long way for businesses. Such a funnel can be easily set up via Google Analytics goals or by enabling conversions in GA4.
A concrete example could be turning these steps into GA goals:
Pageview (sales landing page) → add to cart button click → purchase
After weeks of knowing what conversions you hit on average, you’ll be able to quickly identify bottlenecks and optimization opportunities.
You should master predicting your conversion rates before jumping into split testing. You can optimize your websites for years this way without A/B testing.
Why am I mentioning forecasting?
Because A/B tests can still achieve good results for the micro conversions of websites with low traffic and low conversions. Therefore, using the GA goals (or conversions) setup we’ve just shown you will include the in-between steps (i.e., add-to-cart button click).
💡 Top Tip: Low-traffic websites can still leverage A/B testing by testing macro conversions. Figure out the funnel steps that you will enable as goals or conversions in Google Analytics.
Number of Traffic and Conversions
Many people often ask how much traffic is needed to run A/B testing.
Marketers do not have a conclusive answer. However, there are ranges and other factors to be taken into account for your test to be reliable or statistically significant.
You will need approximately more than 10,000 monthly visits to your site minimum. Additionally, you’ll need a minimum of 100 to 500 conversions per month.
Still, there’s more to consider. That’s why calculators like this one solve these issues.
But again, the pro tip we shared with you about tracking your funnel steps in the previous section can help you obtain results if your site has low traffic.
We’ll also provide you with detailed solutions in the analysis section at the end of this post.
How to install Google Optimize?
To start, we’ll create a Google Optimize account and container that will be linked to Google Analytics. Afterward, we will install the Optimize code snippet either manually or through Google Tag Manager.
We’ll show you both and explain which method to use for your situation.
Using your Gmail account, go to this link and click on Get started.
Choose your account settings and remember to always get permission from your client or company before checking mark Benchmarking.
You will then select your preferences for emails sent to you related to Google Optimize.
Your account and container are now created. You’ll land on a page inviting you to create your first experience.
Since we should install the snippet on our website, let’s click on the settings icon instead.
A slide-in popup will appear with your container settings details. Here you can find four items:
- Your Google Optimize container name and ID
- The linking feature to your Google Analytics property
- The Optimize code snippet to install on your website
- The Chrome extension tool required to use visual editor (more on this later)
First, name your container.
Simply select the edit button and give it a meaningful name.
Containers work the same as those in Google Tag Manager. Therefore, if you have multiple websites, you can create a new container for each.
To link your Google Analytics property, click the Link to Analytics button.
A popup will slide on your screen. There you can choose a property to link to Optimize.
It’s advisable to choose your day-to-day property since you can only link only one property. If you’re still using the previous version of GA, the option to select a Universal property and a view will be available.
Google Optimize provides different options and limits for each version. You can take a look at it here.
We will choose a GA4 property. Now click on the Link button.
Great! You’ve connected your Google Analytics property. We’ll now automatically be brought back to our Containers settings in the first popup to continue our setup.
Now it’s time to install Optimize to your website. We’re now in the Setup instructions section where you’ll find your Google Optimize snippet.
There are two ways to install Optimize:
- hardcode one of the Optimize snippets in your website
- use Google Tag Manager.
Note that the first method is considered best practice. However, there may be times when this method isn’t possible. This infographic explains which method you should use in each situation.
You may wonder what to pick between the synchronous and asynchronous snippets. To simplify things, 90% of the time you’ll use the synchronous snippet (optimize.js).
In addition, this snippet is recommended for most users in the Optimize Resource Hub. This is the same snippet that you’ll find by default in your Container settings under Setup instructions.
If you work with a client, you must read the documentation to see if the other snippet (asynchronous) may be a better option.
Regardless of which code snippet you choose, the location where you’ll hardcode it in the source code of your website matters.
You can copy the code by clicking on the copy icon in the Setup instructions.
The code must be placed at the top of the <HEAD> on all your web pages. By adding the Analytics code in your header once, the code will be made available throughout all other pages.
Some website builders like Squarespace allow you to place the code directly in the head via a code injection feature.
In the Container settings, the last piece of the Setup instructions is the Install the Chrome extension. You need this extension to create experiments and to use the Chrome browser.
Snippet Placement Exceptions
🚨 Note: There are exceptions when it comes to the snippet placement. For example, if you have a Data Layer script on your site, then the Optimize snippet must come after the Data Layer script.
Any of the following must be positioned before the Optimize snippet:
- The Data Layer
- Any scripts that declare JavaScript variables, functions, or set cookies
- The anti-flicker snippet (optional)
The anti-flicker snippet is optional, but it’s best practice to place it for clients as a precaution.
A page flicker is when a visitor lands on a page and the content quickly changes to display something different.
This happens often with A/B testing. When the page loads, the initial version (page A) changes to show the second version (page B).
Not only this is bad when it comes to user experience, but it can also cost you money since users may sometimes have to wait for the page to load or be presented with a page where conversions may not be successful.
The anti-flicker snippet prevents this from happening.
If you decide to use this snippet, copy the code here. Don’t forget to replace the container ID with yours. This applies also if you deploy it with Google Tag Manager.
Let’s visually see the best order for scripts/snippets placement.
How to Install Google Optimize With Google Tag Manager
What if you can’t access and manipulate the <HEAD> tag of your website? Enters Google Tag Manager. If you’re already a GTM user, you’ll notice that the interface of Optimize is quite similar.
Let’s go over the steps to install Optimize with Google Tag Manager.
Back in our Container settings, copy the Optimize ID.
In Google Tag Manager, go to Add a new tag.
Now follow these steps: Tag Configuration → Choose tag type → Google Optimize
Paste your Optimize container ID in the space under Optimize Container ID, and in the next section below in Triggering, choose the All Pages trigger.
Save your tag and let’s test it using the Preview Mode. If you’re not familiar with setting up tags and triggers, then take the time to read our Google Tag Manager tutorial for beginners.
We can see that our Google Optimize tag has fired.
Back in your workspace GTM, submit your changes, and make sure to put a meaningful name in the Version Name. Click Publish.
Creating an Experiment in Google Optimize
Now, let’s create our first experiment. We’ll create an A/B test.
Close the Container settings page.
You will land on a page inviting you to create your first experiment. Click on Let’s go.
You will be prompted to name your experience. We recommend that you use a name related to what you intend to test.
For example, if you’re going to change the header of an XYZ page, your experience’s name could be “header-xyz”.
Add in the page you are going to use for your experiment. Simply paste the page URL in the space under What is the URL of the page you’d like to use?
Lastly, you can select the A/B test in the section What type of experience would you like to create? Then click on Create.
At this point, you’ll be directed to the page of your experiment. There, you’ll find all the details and additional settings specific to your test.
Click Add variant.
This variant is the new version of the original page we’ve entered before, except that it’ll have the modifications we want to test. This is the ‘B’ of the A/B test.
Name your variant. And click Done.
In our case, we want to replace the text of the search button with an offer. The search button has a text saying “Search”, and we’ll replace it with “Get a 20% discount.”
This is just for demonstration purposes. If you’ve followed our instructions on forecasting micro and macro conversions, then you’ll have a better idea of what to tweak.
Let’s go back to our experiments page to look at the other settings.
Targeting and Variants
We’ll focus on two features in targeting and variants: weight and edit.
You probably noticed the default weight proportions of 50% distributed to each variant. But what does it mean? Weight is the amount of traffic that you decide will go to each variant.
The default of 50% means that there’s an equal amount of chance for visitors to see one page or the other.
🚨 Note: Weight proportions can impact your sales and other marketing efforts. Consider the impact of one variant receiving more traffic over some time.
Weight can affect your sales and can impact your affiliates since traffic can be sent on a page with modifications that may turn away visitors.
If you’re not sure how to distribute your weight, you can use this rule of thumb: 75% for the original and 25% for the variant.
You can adjust these proportions whether you select the weight of the original or the variant. Simply click on weight and then adjust your proportions.
Then in Edit variant weights, click on Evenly split and select Custom percentages. Type in your percentages.
We can have some fun now as we edit the visual of the page of our variant.
Here is where the Chrome extensions come in handy.
You will land on your original page, but you’ll notice a bunch of HTML references and an Edit palette.
It’s here that we’ll modify our variant. Remember, we want to modify the search button by replacing its text with a discount offer.
So, let’s right-click on the button.
A dropdown list of options will show up. Because we’re only interested in changing the button’s text, we will select Edit text.
You can select the options that suit your needs and skill level.
You can modify whatever you want. For example, you could change the color and size of the button as well, by scrolling down the Edit element palette in the RGB field.
Now that we’ve changed the text, this is what our button looks like now:
If you’re happy with the result, click Done. The button can be tricky to find. It is located at the bottom right of your page.
At the top right, click Save.
Back to our experiment’s page. Let’s continue our settings walkthrough.
Our button needs to be available on our Demoshop website. So, wherever visitors navigate, this button must appear.
We can ensure this by changing the settings in Page targeting and clicking the Edit icon.
We can use the following configuration to make sure our changes remain available on all pages. Let’s use the match type: URL and Contains.
🚨 Note: If your changes must remain only on individual pages, then don’t change the settings.
In Targeting and Variants, the last setting is Audience targeting. We will not use it for this tutorial, but you should definitely have a look.
Audience targeting allows you to show your variants to different groups of users. These can be users coming from different campaigns (i.e., UTM parameters), devices, geography, etc.
Measurement and Objectives
Lastly, we’re going to optimize for our objectives.
Objectives are metrics you want to improve. They are essential to assess the performance of your variants and determine which one is the winner. They equate to goals/conversions in Google Analytics.
Objectives for a lead generation site can be form submissions or revenue for eCommerce sites.
This is the reasoning behind having goals or conversions enabled in Google Analytics. Don’t worry if you don’t have any in GA, since Optimize makes it possible to configure them within its platform.
There are 3 types of objectives proposed by Optimize. These are system objectives, Analytics goals, and custom objectives.
System objectives are common goals found across industries such as PageViews, revenue, AdSense revenue, and more.
Google Analytics goals are those you configure in Google Analytics. You will find them in the list of goals in Optimize.
Custom objectives are those you can configure within Google Optimize. They are useful if you don’t have them set up in GA.
To access or set up your objectives, go to Measurement and objectives, then click Add experiment objective, and select Choose from list or Create custom.
In our case, we’ll select Choose from list. The following area is where you can select your objective. Here you find Optimize system objectives and your Google Analytics goals, as well. Here, we chose Pageviews.
Description and Checking of Your Installation
We strongly recommend that you add a description of what your test is about. This is best practice, especially if you run multiple experiments or work for different clients.
Simply go to the Description area and click the edit icon to add your description.
Starting Your Test
Lastly, you can’t launch your test without verifying your installation. On the same page, go to Settings.
Back to your main page, scroll up, and click Start to begin your experiment.
Congratulations! Your test is now finally running.
How Do I Analyze Google Optimize Results?
Three main factors support the analysis of your results: time, conversions, and the p-value.
First, let’s locate all three of them before we look at how they work together.
These can be found in the Reporting section of your page at the top left corner.
Time-related results are given in the form of a message at the top left corner right under Reporting. Here the message says ‘Optimize experiments need to run for at least two weeks to find a leader’. Google Optimize recommends two weeks.
Your conversions are displayed under the second Experiment column. Since we selected Pageviews in our objectives, we’ll look at the conversion results under ‘Experiments Pageviews’.
🚨 Note: The numbers 1046 and 1640 are not the number of PageViews for each variant. They are the number of conversions.
Lastly, to make sure your results didn’t occur by chance, you need the p-value. You can find it under the Probability to be Best.
We know where our analysis factors are and at this time, we can learn how they come together to identify a winner.
Time
💡 Top Tip: Allow your experiments to run for 7 days.
Experiments have time considerations. A test that runs for too long will cost resources and take up time that the company could use to optimize areas for more immediate results.
On the other hand, if the test is too short, your results won’t be reliable.
A good rule is for your tests to stretch past a period of 7 days, not less. Google Optimize recommends a minimum of 2 weeks, it’s not the most optimal but proceed carefully. More than 3 is definitively reliable.
The point is to avoid relying on results that are less than 1 week, no matter how good the conversions or p-values are.
Time also includes audience behaviors.
If, for example, your test was running during a holiday then you should run the test for one additional week.
P-value
The p-value tells you if the results of the test happened by chance or from your modifications.
Aim for a 95% (meaning there’s only a 5% probability that the variant won by chance). Lower p-values such as 90% or a bit lower can work. But this depends on the level of risks you’re willing to take.
Also, if your p-values are close to each other, Google Optimize is informing you that there isn’t much difference between your original and variant. Therefore, that promising headline or that new button color isn’t going to have much impact. It’s really up to what you prefer.
There is a visual representation of the p-values on the right side under Modelled Pageviews per Session. If the boxplots overlap or are far apart, they will reflect that distance.
How Do They All Come Together?
You can safely proceed with your changes when the success requirements we discussed above are met for each factor.
To illustrate, a test is considered reliable after running for a little more than 2 weeks, with more than 100 conversions and a p-value of 95%.
However, if one of the factors is below the success requirements, the test would be deemed unreliable.
Using the previous example, the experiment would not be reliable if any of these conditions occurred individually:
- less than 100 conversions
- less than 1-week test
- p-value inferior to 85%
If any of one of these occur, you need to wait a little longer before making any decisions.
How to Use Google Analytics and Optimize
Google Optimize declares which variant is the winner. Whereas Google Analytics provides insights about that winner.
Remember that optimization is an ongoing process. Today’s winning variant can also negatively impact the subsequent steps of your funnel over time.
This impact can be monitored by looking at your goal/conversion funnels in Google Analytics.
Here’s how this works. The winning variant of a lead magnet landing page helped increase conversions for subscriptions.
Later on, if the subscription goal conversion rate declines, you’ll have to rework that lead magnet landing page.
Another Google Analytics advantage is that you can apply segments to your experiments and use them for retargeting.
Summary
By now, navigating through Google Optimize should not be a mystery to you.
We’ve covered everything you need to know about A/B testing. We’ve also equipped you with the tools, frameworks, and strategies to set up your tests and analyze results like a pro.
Learn how to monetize your analytics skills with our handy guide on how to make money selling analytics services.
It’s time for you to run your experiment. Let us know how it goes in the comments below!
