Google Content Experiments: Pros and Cons

by Colin Temple

Last Friday, Google announced that it’s canning Website Optimizer, its website testing tool, in favour of a new one called Content Experiments. Unlike Website Optimizer, Content Experiments is not a stand-alone product, but rather a new feature of Google Analytics. The new tool let’s you do A/B (/C/D…) testing by swapping out different URL-based pages for one another. I thought I’d share my thoughts on the change.

Why it’s awesome

  • Because it’s a part of Google Analytics, all of your goal conversion tracking will work natively with Content Experiments. You define your experiments to work with GA goals.
  • For the same reason, the setup is easier — you only need to add code in one place, on the ‘original’ page, which is an improvement over Google Website Optimizer.
  • This also opens up the potential to pull experiment data from the Google API, which would be great for ongoing performance reporting.

Why I’m still wanting

  • The change is great for the reasons mentioned above, but there isn’t much here capability-wise that wasn’t in Google Website Optimizer. You can easily compare different pages if they have unique URLs. That’s great, but what about variations in a page with the same URL? We often want to check variations of product and checkout pages, pages which often can’t easily have multiple URLs. There’s still quite a bit of custom work required to accomplish this with Content Optimizer as it is, since you’ll need to write server-side code that responds to a GET variable in the URL (ie. ?variation=3). For those working with a somewhat inflexible shopping cart software, like many online retailers, this isn’t always an option.
  • Another annoyance is that you can only use URL and Event-based goals in order to track conversions from each variation. You can get ecommerce transaction revenue amounts through goals when you use a thank-you page and set your goal value to 0 — but I’d prefer to link my success to transactions themselves, so you can see more information on what products were purchased (and so forth) using each variation.

Overall, I think the consolidation of these two tools — Website Optimizer and Analytics — is a great move and a step in the right direction. Hopefully this is just the beginning of new features for Content Experiments that will give Google, and Google Analytics users, an edge in the testing space.

If you want to try out the new tool, it’s located in the Standard Reporting area of Google Analytics under Content > Experiments. Meanwhile, Google Website Optimizer is set to close its doors on August 1.

Cheers,
Colin

Colin Temple

VP, Product

Colin serves as VP, Product for Napkyn Analytics. A diverse background in data, software and marketing and an education in logic and philosophy give Colin a unique perspective on where the analytics practice is, and where it should be.

See more posts from Colin