How to A/B Test with Google Tag Manager

A/B Testing can be done with Google Tag Manager. Why? Because Testing suites like VWO or Optimizely can get very expensive, very fast.

In this video, you are going to learn how to build an A/B Test right into GTM and send the data to Google Analytics.

The Next Web Post: http://thenextweb.com/voice/2016/03/03/marketing-the-tnw-way-9-ab-testing-with-google-tag-manager/#gref
A/B Testing Script: https://github.com/thenextweb/cro
Test Significance Calculator: http://getdatadriven.com/ab-significance-test
GTM Ressource Guide: https://measureschool.com/guide

In this video, I’m going to show you can use Google Tag Manager to conduct simple A/B Tests and measure the results within Google Analytics. All the more, coming up right after this. Okay, today our journey starts at this next Webpost by Simon Vreeman. Together with Martijn Scheijbeler, they developed an A/B testing solution for The Next Web in order to run A/B Tests through Google Tag Manager on their website. 

Now, this is a really large tag blog and it makes sense to utilize the built-in capabilities of Google Tag Manager to do A/B testing for free. So if you want to find out more about that solution and how to use The Next Web, check out this post. They were also generous enough to write up their A/B Testing solution in a GitHub repository where you can find the actual code that we will implement today. But, let’s start at the beginning. We want to implement an A/B Test on our Demoshop here. We already have Google Analytics installed through Google Tag Manager and we now just need to find a test subject. So, I have chosen to… actually look at this product page. We have a sale button here. What if that sale button would be a different color? Since we are dealing with Google Tag Manager and Google Tag Manager is based on JavaScript, you would need to know how to manipulate this element with the help of JavaScript. So you would need to know a little bit of JavaScript at least in order to know how to manipulate elements in the dom with the help of JavaScript. 

So let’s try this out. How would we pretest our manipulation? We can go into the developer tools which we’ll find under View. And then JavaScript Console. And we can select this element by right-clicking on it and inspecting this element. And we see in our JavaScript Console, that we are dealing here with a Span Element with a class of OnSale. Now, we can go over to our JavaScript Console and select this element with the right commands. And now okay, so it would be “document.getElementsByClassName.” then entering the class name “OnSale” and we see that we return a correct statement here which selects our element. Now since this is an array, which was the first element, and now we want to change the style of this element. And specifically, the background… to something that we can choose, like green. Let’s press Enter here. And we can see the element changes. And this is the command we will use in order to execute our A/B Test. Now the actual logic of when to change the color will be done by an A/B testing script and we will use that script by the guys of The Next Web. 

So let’s implement our A/B Test through Google Tag Manager. Scroll over to Google Tag Manager. The first step would be to actually activate a built-in variable. Let’s go over to variables, and configure our built-in variable and look for the Random Number Variable. This is a variable that will randomly pick a number and we can use the variable in our A/B Testing script because then Google Tag Manager will decide randomly which version of the A/B Test to user will see. So, once we have activated that, we can go over to tags and implement our A/B Testing Tag. Let’s give this all a name. This is a custom tag and it will execute our A/B Test on the product pages. Now as a tag configuration, we’ll go with our custom HTML tag, and we can implement the code from the code repository here. We’ll just go to Raw and then mark this all, copy it, and before we paste it into our custom HTML tag, we actually need to surround it by script tags. And then we can paste it in… and we now have a fully built testing solution. 

What will this actually do? Well first of all, it will execute our A/B Test, it will also set a cookie so if the user has already seen one or the other version, he will continue seeing that for the duration of the test. You can also set an expiry date and whatever version the user sees, will be pushed into the data layer. So there, the data layer pushes within. Now all we need to do is implement our execution of what we want to happen once a variation should be shown. So let’s go down here. First of all, we have here prefix, this is for the data layer. We can change it over. In this case, it’s The Next Web. Let’s call this Demoshop. You see here that it uses our random number variable and then we have here the changes that we can implement. Now, you can do multi-variable testing. You can also test multiple variants at the same time, but we just want to do a simple A/B Test so we’ll delete our second variables here. And just in this column, we need to input our function that we want to execute once the variant should be shown. So we already prepared this, so let’s go back. We have here our code. We can copy it again. Put it into our execution function so it will execute once our variant should be shown. Alright, that’s it. Now we can go ahead and configure our trigger. And we just need to choose the side that we want to have this firing on. In our case… In our case, we want this just on this Flying Ninja page for now. So let’s copy just the URL. And input that as a Page View Trigger. As early as possible, only some page views. We’ll go with the URL path. Contains our URL here. Let’s give this a proper name. So fire on a Page View and fire on our Flying Ninja page. Alright, let’s save this. Save our tag and try this all out. Let’s go into the Preview and Debug Mode. Which you can also reach by this button here. And this will put our browser into the right state. Let’s close our developer tools and we see our site reloaded and it already fired the first A/B Test because we can see the A/B Test event here in the data layer. So, let’s look into the data layer. And we can actually see it was something was pushed here and it actually shows us the version that was shown. In our case it’s zero which means this is the actual control and not the variant. 

Now if I reload this page, nothing really happens and we get the same variables again. That’s because we have a cookie installed that will ensure that we won’t see multiple, different, other variations and then screw up our test. So, we’ll be able again to be able to access our developer tools in order to get rid of that cookie, just to test it out. Under Applications, we have our cookies here, and we see a cookie is stored with this ds-cookie or whatever you have as a prefix that you can delete. Just to see if it works. Let’s reload this again. Unfortunately, we’re getting again. Apparently the, yeah, control version. Let’s delete this again. Try it out again. No, no luck this time. This can take a while. And here we go. We see that our code executed and this time we got the green button. Now if you reload this again, you can also see the big downside because Google Tag Manager is actually loading asynchronously so you actually see a little bit of a flash effect once you reload this page. But our A/B Test works on that basis and if you’re okay with these changes that are made, we’ll be able now to analyze the results. 

Now, all we need to do is capture data and analyze our results and this can be done with the help of Google Analytics. So in Google Tag Manager, you already saw that there was an event pushed into the data layer and we just need to pull out the data of the right variant and transfer that data to Google Analytics in order to later then see which users converted based on our changes. So how do we do that? First of all, we need to prepare a data layer variable to pull out this value. So let’s go over to Google Tag Manager, under Variables we can build a new user defined variable. This will be our data layer variable of type event action. We’ll go with the variable configuration here and choose our data layer variable and enter the key that is here Event Action. Let’s save these. By the way, you could also recreate the event category and the event label. I will leave that out because these will static for us. But, if you want to fill them dynamically, then you could be recreating those as well. Now, all we need to do is send out our data to Google Analytics and we’ll do this by a Google Analytics Event Tag. Let’s take as a tag configuration, we’ll choose Universal Analytics. I have already a tracking ID saved in a constant variable. So, we will use that, and as a Track Type, we’ll choose Event. Now we can utilize our event variables. For the category, I haven’t created one, so I’ll just hard code it with A/B Test. Then, the action should be dynamically filled with the variant that was shown. So we can choose event Action and if you don’t want this to affect our bounce rate, then we change the Non-Interaction Hit to True. 

Now, if you want to be more sophisticated with your A/B Test, you can even set a custom dimension to make sure that your test results get stored on a user basis in a custom dimension. But, we won’t do this for now. We’ll go ahead and build a trigger and this trigger will be based on the custom event that is sent to the data layer which is this A/B Test. So, this will be a custom event and we’ll file under A/B Test. Let’s configure this. Choose our Custom Event here and as the Event Name we’ll go with our A/B Test. This needs to model exactly what it says here in the event key. Let’s save this. Save our Google Analytics tag and refresh our preview in demo mode. Let’s go back to our page. Refresh it there as well. And you see that we’re getting the first variation here. Our A/B Test data layer event fired and based on that, our Google Analytics event tag fired. And transferred that data over to Google Analytics. So, we should be able to see that also in Google Analytics itself in the Real-Time reporting. We can go here to Events and there was just an event that happened. Let’s go over to last events, because this is a non-interactive event. And we see our A/B Test with the Event Action ds-1-1. So we have here, our variation number one. So, anytime somebody sees a different variation, this would fire an event. Now what can we do with this data? Within Google Analytics you can obviously look at how many people saw different variations variations under the Behavior Report. So, this is the different variations that we saw. And now you can match that up with actual conversion data if you have for example an e-commerce shop. It would be good to capture conversions on a purchase basis. You could also build in a goal for a button click. And that’s what we have done previously. So, we have defined our goals correctly and you can build a custom report. That shows you your A/B Test results based on how many people saw your version. And then, how many people clicked on your Add to Cart or purchased a product. Now this report is not yet filled because it takes a while to populate once you have sent data in. Obviously, you would also need to make sure that your test is actually significant and see which variation actually has won. There are multiple calculators out there that can do this for you. You can enter your visitors to each version, the overall conversions, you’ll get conversion rates, and your results that will tell you how much better the conversion was, but also the significance level that lets you know whether your A/B Test is actually valid. Don’t forget at the end, if you want to publish this to your users, then you can go ahead and publish a version in Google Tag Manager. Give this hole a name. And your test will be live for your users. 

So there you have it. This is how you can build a simple A/B Test with the help of Google Tag Manager. Really, a great way to A/B Test your site in an affordable manner with Google Tag Manager and Google Analytics. If you want to find out more, check out the resources in the description below and if you like this video, please give us a thumbs up and subscribe to our channel because we’ll bring you more of these tutorials every week. My name is Julian. ‘Til next time.

JOIN US!

Master Data & Analytics with Measuremasters

Exclusive Courses & Workshops | Ongoing Troubleshooting | Support Resources, Tools & much more
guest
0 Comments
Inline Feedbacks
View all comments

now it's time to

Start measuring like a master

Itching to jump into the world of MeasureMasters? This is what you have to look forward to.