Deploying Optimize 360 Through Google Tag Manager Versus Direct Deploy

by Peter Adamson

Over the past few months, Napkyn has helped clients get up-and-running with Google’s new A/B testing tool Optimize 360. In our experience so far, it is a pretty great tool for targeting and optimizing experiments primarily because it features an intuitive interface and plays well with other tools within the marketing suite.

Optimize 360 can be deployed either through Google Tag Manager (GTM), or as part of the Google Analytics (GA) direct deploy snippet, and this choice is what our clients tend to have the most difficulty with because either will work but both have trade-offs.

Google recommends deploying Optimize 360 directly in the GA snippet, which allows Optimize 360 to initialize faster reducing overall page load time and improving user experience. The flip-side to this is that you lose much of the convenience that GTM provides for deploying GA, which is a significant factor for highly customized/complex GA deployments and for testing teams that frequently update their GA deployments.

So, if you have GA deployed through GTM with just the basic settings, and no custom dimensions, metrics, or special trigger conditions, choosing the direct deploy snippet for the sake of Optimize 360 would be fairly straightforward requiring some code changes onsite, work in GTM, and thorough QA afterwards. But if you have, for example, a GTM/GA setup with dozens of custom dimensions, some populated with lookup tables and customJs variables, different reporting properties for different pages/sites/environments, plus a dataLayer setup to report Enhanced Ecommerce on pageviews all configured in GTM, migrating this out of GTM and directly into the site code is going to be a major undertaking. 

The Experiment

With Optimize 360’s load time being the key difference between the two deployment options, I thought it would be worth showing exactly what the difference is in practical terms.

I set up two demo pages each deploying the same Optimize 360 experiment from the same Optimize 360 container. The only difference between these two pages is that one is deploying Optimize 360 through GTM, and the other is deploying Optimize 360 as part of the direct deploy GA snippet.

The Optimize 360 experiment takes this original page and applies some rather obnoxious style changes to 99% (the largest weighting available in a single variant test) of people visiting either the GTM deployed Optimize 360 page or the direct deploy page.

ab test experiment example

The Setup

Both of these pages have the deployment as early in the HTML as possible (optimized for Optimize 360), but also include a simple dataLayer declaration and the recommended content hiding snippet (async-hide) that Google provides to support Optimize 360. Both the dataLayer and content hide snippet precede the GTM or GA portion that subsequently loads Optimize 360.

The HTML/JavaScript for the GTM version is this:

<html>

 <head>

    <meta charset=“utf-8” />

    <!– start combined aync-hide + GTM snippet –>

       <!– dataLayer declaration –>

       <script>

          dataLayer = [{

             ‘pageType’: ‘home’,

             ‘userId’: ‘123456789’

          }];

       </script>

       <!– start async-hide snippet (recommended)  –>

       <style>.asynchide { opacity: 0 !important} </style>

       <script>

          (function(a,s,y,n,c,h,i,d,e){s.className+=‘ ‘+y;h.start=1*new Date;

          h.end=i=function(){s.className=s.className.replace(RegExp(‘ ?’+y),)};

          (a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;

          })(window,document.documentElement,‘async-hide’,‘dataLayer’,4000,

          {‘GTM-WCK3CFR’:true}); /* set to listen for GTM container */

       </script>

       <!– end async-hide snippet (recommended)  –>

       <!– start google tag manager GTM-K9BSZNM –>

       <script>

          (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({‘gtm.start’:

          new Date().getTime(),event:‘gtm.js’});var f=d.getElementsByTagName(s)[0],

          j=d.createElement(s),dl=l!=‘dataLayer’?‘&l=’+l:;j.async=true;j.src=

          ‘https://www.googletagmanager.com/gtm.js?id=’+i+dl;f.parentNode.insertBefore(j,f);

          })(window,document,‘script’,‘dataLayer’,‘GTM-WCK3CFR’);

       </script>

       <!– end google tag manager GTM-K9BSZNM –>

    <!– end combined async-hide + GTM snippet –>

   

    <!– REST OF THE PAGE HTML FOLLOWS –>


The HTML/JavaScript for the
direct deploy version is this:

<html>

 <head>

    <meta charset=“utf-8” />

    <!– start combined aync-hide + GTM snippet –>

       <!– dataLayer declaration –>

       <script>

          dataLayer = [{

             ‘pageType’: ‘home’,

             ‘userId’: ‘123456789’

          }];

       </script>

       <!– start async-hide snippet (recommended)  –>

       <style>.asynchide { opacity: 0 !important} </style>

       <script>

          (function(a,s,y,n,c,h,i,d,e){s.className+=‘ ‘+y;h.start=1*new Date;

          h.end=i=function(){s.className=s.className.replace(RegExp(‘ ?’+y),)};

          (a[n]=a[n]||[]).hide=h;setTimeout(function(){i();h.end=null},c);h.timeout=c;

          })(window,document.documentElement,‘async-hide’,‘dataLayer’,4000,

          {‘GTM-PKZB6GX’:true}); /* listens for Optimize container */

       </script>

       <!– end async-hide snippet (recommended)  –>

    <!– Google Analytics with Optimize snippet –>

    <script>

       (function(i,s,o,g,r,a,m){i[‘GoogleAnalyticsObject’]=r;i[r]=i[r]||function(){

                (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),

             m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)

       })(window,document,‘script’,‘https://www.google-analytics.com/analytics.js’,‘ga’);

       ga(‘create’, ‘UA-63196131-11’, ‘auto’);

       ga(‘require’, ‘GTM-PKZB6GX’); /* Require Optimize Container GTM-XXXXX */

       ga(‘send’, ‘pageview’);

    </script>

    <!– end Google Analytics with Optimize snippet –>

    <!– end combined async-hide + GTM snippet –>

    <!– REST OF THE PAGE HTML FOLLOWS –>

These are very similar. The only difference being one has a GTM snippet and the other a GA snippet.

The Results

So let’s take a look at the resulting page performance:

results from experiment showing pagespeed against request

The gtmetrix.com side-by-side comparison confirms overall direct deploy page performance is marginally better than GTM, a difference in the range of 1% overall and 15kb in total size. 15kb would be about the size of the GTM container (17kb), which the direct deploy page does not have, so this is all what I would expect. I ran this comparison 5 times with the default settings, and the ‘Fully Loaded’ time had a variance of about 0.5 seconds for either page. The GTM version loaded the page between 1.2s – 1.8s from test to test, and direct deploy loaded between 1.2s – 1.7s.

But that is all too general. I want to know about Optimize 360 specifically, and when it is requested by the page and when it is loaded. Looking closer at the resource requests generated from the two pages the differences in Optimize 360 performance is a little more significant:

results from GTM version of the experiment

results from direct deploy version of the experiment

These images show the resource requests coming from the page, and when they occur relative to other resources. Like the ‘Fully Loaded’ time, timings will vary from test-to-test but what remains consistent is that Optimize 360 from GTM tends to be requested and loaded much later relative to other resources. This example shows the most extreme difference I saw, among the tests I ran, with Optimize 360 from GTM being requested 1 full second after the direct deploy version.

So, the initial request to load Optimize 360 would happen between 0.4s and 1.0s later from GTM ( relative to the start of the page load). The amount of time it would take for Optimize 360 to fully load was pretty much the same and always under 0.1s.

I ran another test, this time throttling the the network to ‘Cable’ speeds (1 Mbps)  to see if making the bandwidth a little more consistent impacts the results:

results from direct deploy version of the experiment

results from the gtm version of the experiment

The results here are consistent with the uncapped test. The only difference is everything takes about twice as long to load. The GTM-deployed Optimize 360 gets requested about 0.5s to 1s later than the direct deploy version, and is consistently among the very last resources requested by the page.

The reason for this consistently later request for Optimize 360 from GTM is the GTM container (gtm.js) must completely load then deploy the Universal Analytics library (analytics.js), which must also completely load, before Optimize 360 is even requested. The direct deploy Optimize 360 also needs to wait for Universal Analytics to fully load before the Optimize 360 request is sent, but the direct deploy method starts one step ahead.

The GTM delay will also vary depending on the size of the GTM container. GTM is well optimized for size, and the one in this example is bare-bones at 17kb. But containers can easily grow in size to 20-30kb after a few dozen tags are added along with their supporting triggers and variables. This container size variance in most cases is only a factor on the landing page as GTM, Universal Analytics, and Optimize 360 resources will cache (unless this is disabled in the browser), making load times on subsequent pages pretty much instant, but the sequence of one tool requesting the next still has to happen.

Conclusion

Before doing this I kinda knew what to expect in terms of results, but having actual solid numbers about the performance difference between Optimize 360 deployment through GTM vs direct deploy is really useful.

Knowing that the GTM deployment will add up to a second to the time it takes for Optimize 360 to work, when compared to direct deploy, enables a more informed decision of what deployment option is the right one for our clients, or anyone else making this same choice.

Peter Adamson

Implementation Specialist, and Google Tag Manager Practice Lead

Peter is an Implementation Specialist and Practice Lead for Google Tag Manager. He partners with our enterprise client development stakeholders to help guide them through and assure the quality of their most sophisticated analytics infrastructure requirements.

See more posts from Peter