Managing Limitations: A Brief History of Web Analytics

by Jim Cain

Web Analytics, while still very much an emerging discipline, is constantly under intense scrutiny by its practitioners. Not surprising, since the entire goal of the space is about clear understanding and continuous improvement.

In the early 90s, web analytics was not a marketing activity. Rather it was about technical performance of the website, and the reports were generated from logs and owned by the same guys and girls who keep the server room running. At some point in the mid 90’s, Marketing realized that they could use that data to understand the business impact of the website and a discipline was born. This was the golden age of WebTrends, the first marketer facing web analytics tool.

Because of the fact that this software was built by IT, the unit of measurement was around hits, or individual calls to the web server. 10 images on a page would be 10 hits. Not the richest data for business analysis. An analyst could discover which elements of a site got the most ‘hits’ but could not tie them to types of visitors or business goals in such a way that they could affect change.

In the late 90’s new vendors like Omniture, Coremetrics and WebSideStory entered the analytics space (as well as Urchin, DeepMetrix and IndexTools, which would later become Google, Microsoft and Yahoo Analytics). These vendors built products around marketer requirements rather than IT, and this gave rise to both page tagging (removing the main limitations of log analysis), and richer reports designed for Marketers.

This period from 1997-2000 (when the first dotcom bubble burst) was the first big vendor gold rush in digital marketing. Many of the bigger companies (and lots of the failed ones) in digital marketing were founded in this period (think Akamai, Offermatica, Lyris, ATG etc.).

The limitations of these types of analytics tools and vendors are significant:

Tracking the Visit and not the Visitor: While tracking visits is a lot better than a server call, it is still not the best base unit for measurement. No one sells to a visit they sell to a visitor, and a lot of high value information is left on the table because of the lack of this kind of tracking. Want to know how many visitors to your site created an account and then came back later to buy? Forget it.

No sharing of data:
Each product has it’s own proprietary reports, data capture and data naming conventions. This means that it was impossible to create a decent report on a full set of digital marketing campaigns. This taught most digital marketers to treat each use of a product as a ‘campaign’, and analyze each campaign as a standalone. This explains why offline marketers will think of a marketing campaign as a multi-message, multi-medium program (think TV ads, radio ads, sponsorships), and online marketers will think of a campaign as a single activity (think one email newsletter, one personalized ad, one coupon code).

No sharing of tags: Each analytics or marketing product had a separate deployment on each client site, meaning that most websites would have several javascripts running all basically performing the same task for different products. Also if a page is partially loaded (visitor clicks through or off the page quickly), not all of the javascripts will run meaning that there will be a data discrepancy between the analytics tools running on the page – messy stuff.

No legitimate ROI calculations: As mentioned in other blogposts, all vendors sell on a direct ROI value proposition that is almost unprovable because of the prior two points. The only things you can do are to trust the vendors reports, or to put together a complicated an inefficient way of ‘guesstimating’ impact in the analytics data (i.e. merchandise dynamic upsell/cross-sell tools)

The last ten years (2000-2010) has been less around building tools than trying to align the tools into a discipline. The majority of standout efforts in Analytics have been around the rise of the practitioner, a person who has dedicated their career to doing nothing but mastering the tools and tricks of online performance management, web analytics and digital optimization (huzzah!).

There are a number of talented and smart people who were in the vanguard of web analytics, but for the purposes of not listing all 250 of the smart people I follow on twitter I will break them down into four rough categories (note there is a ton of overlap here, but these groups divert enough that the naming convention works):

The WAA: Started by Jim Sterne and Bryan Eisenberg in 2004, the Web Analytics association has roughly 1500 global members and represents the guiding light in the field of digital data analysis. A wholly volunteer driven enterprise, the WAA is chock full of members who contribute actively on a daily basis to drive forward our industry for Standards, Education, Certification, Research and Globalization.

WebAnalyticsDemystified: Started by Eric Peterson in support of his first book of the same name, WebAnalyticsDemystified has become a web portal for analysts that rivals (in some cases on purpose) the WAA for mindshare on some of the key topics in the industry. Ever been to a Web Analytics Wednesday? Thank WebAnalyticsDemystified. This site is now a full consulting business that includes some of the brightest minds in the industry.

GrokDotCom and the Eisenbergs: Despite some major changes in the last year, FutureNow’s GrokDotCom website spent most of the last 10 years as the online resource for conversion optimization analysis, the critical driver behind the adoption of analysis in most organizations (i.e. if we can understand it, we can grow it). Founded by Bryan and Jeffrey Eisenberg, this blog, and several amazing books they wrote on the subject have acted as the foundation behind the continuous improvement discipline in analytics. These guys are now running their own private consulting company, and a lot of their best practices have been captured for the practitioner at Market Motive.

Analytics Tool Vendors: Producers of web analytics software have always promoted the industry as a whole and their strengths in specific. When Urchin became Google Analytics in 2006, the focus of the promotion moved from the corporation to the individual – the evangelist. Avinash Kaushik is a force of nature in the web analytics community, a great supporter of why and what you should measure – and huge for the adoption of Google Analytics. Other tools vendors have their own evangelists, the standouts being Dennis Mortensen at Yahoo Analytics and Akin Arikan at Unica.

The thing to take about this recent period is that because the discipline was catching up to the technology, practitioners were forced to learn how to do analytics around the limitations of the tools they were using. Figuring out how to make decent reports and analysis around visit level tracking, fighting the ongoing battle with cookie deletion, trying to get multiple reports from multiple vendors to work together…’s no wonder there are only a few thousand full time web analysts….

Frankly some types of analysis, especially attribution and vendor ROI, are like writing about a solar eclipse in grade 3 science class – don’t look at it directly and then talk about what you think happened.

2010 and beyond represents a seismic shift in the technology underpinnings of web analysis. This is happening in large part because the disappearance of the key players in the market has created a void that can be filled in by a new generation of companies who have been informed by what analysts want and not what is possible from server room data.

In the last six months, two of the biggest players in the web analytics space have been acquired; Omniture by Adobe, and Coremetrics by IBM. These new parent companies are not organizations that answer the phone from prospects who aren’t Fortune 1000. The old ‘Sharks-Jets’ battles between free and paid analytics tools are now over, and there is a void in the industry right now as analysts and consultants ask “What’s next?”

In my opinion, the change is around putting the analysts and not the technologists firmly in the WA drivers seat – and realizing that clickstream data is not web analytics.

Talk to any practitioner, and she or he will tell you that Omniture/Coremetrics/Google/Yahoo/etc. on its own is about 30% of the total picture of the web analytics universe. They will also most likely tell you that trying to see the other 70% is currently like trying to read underwater.

Enter the next generation of web analytics tools vendors – who focus on helping create a cohesive and complete picture of your WA efforts. This new breed of tools has been informed by analysts and business owners alike, and are designed to work as part of a complete analyst package. For example:

Looking to analyze the lifetime value and onsite behaviors of your paid keyword traffic? Want to see where they move their mouse and how they interact with your shopping cart for optimization? Curious about what they actually think about your business and your site? Want to understand if the paid keyword they clicked on was the most critical part of their marketing experience, or if a banner or an email was what really moved them to make their first purchase?

All these questions can be answered, and tied together into a cohesive report. This just wasn’t possible 2 years ago, even if you were willing to spend several hundred thousand dollars in tech costs.

The next several years will be around reaping the rewards of having the tech companies and the analysts sitting at the same table, on equal footing, answering the questions about how to provision maximum value to their stakeholders. No more managing your work around existing limitations, analysts will be able to be limited only by the quality of their questions and the foresight of their organizations to act on the answers.

What’s next after that? If I could get in the Delorian, pump in 1.21 gigawatts of power, and go to 2015, I think I will see a group of web analysts who realize that their skills would carry through to any other part of the organization; HR,shipping, finance… Web Analytics is the gateway drug for business intelligence in mid-sized organizations.

But I digress.



PS. This blog is the example of the kinds of discussions you will hear when you put a lot of analysts in a room with beer on tap. If you liked this blog, get yourself out to a Web Analytics Wednesday or eMetrics event.

Jim Cain

Chief Innovation Officer & Founder, Napkyn Analytics

Founding CEO of Napkyn Inc., it was Jim’s curiosity about how data can be leveraged by marketers that led to him establishing the company in 2009. He ideated, delivered and iterated all of the early services delivered by Napkyn to enterprise clients. Jim has a unique ability to connect the dots for where the industry and individual brands will and can go with their data-driven initiatives.

See more posts from Jim