Thoughts on WAA's Standards Committee's Web Analytics Definitions Document

I finally had the time today to start poring over this document, released by the Web Analytics Association (WAA) on August 16th. I haven't finished reading all of it in detail yet, but I have these thoughts (well, ok, they're rants) so far.

Let me start by saying that I'm excited about the publication of this document, as it represents a step toward the development of the set of standards for web analytics data collection and data definitions that will be required for true interoperability of web analytics products, and products peripheral to to web analytics that, together, create true business problem solutions. It's a small step, but a step nonetheless. In my dreamland, I see a standards-based library of vertical-specific business events that can be used as the basis for collecting, reporting, analyzing, and integrating analytics data. Web analytics has to move beyond simple counts and ratios of dimensions based on browser/client actions, toward standardized handling and reporting of high-value business events.

So, with that in mind, here are my unfiltered thoughts (rants).

Firstly, I'm left wondering what the purpose of the document is. Is it to put a stake in the ground as to what the standards should be, or is it to act as a sort of "meta user manual" that takes all the variant vendor behaviors into account, attempting to make any standard definition wide enough to include any vendor's product? In cases, it seems to be the latter. Consider this definition of Visit Duration:

The length of time in a session. Calculation is typically the timestamp of the last
activity in the session minus the timestamp of the first activity of the session.

Should not the purpose of a standards document be to declare the standard? Explaining typical behavior belongs in a book about web analytics, not in a standards document. That's not to say that you're going to get all the vendors to agree on the "right" way to calculate visit duration. But if there isn't a standard (or if you aren't declaring one) it doesn't belong in the document (IMHO).

Secondly, I'm a little disappointed that several of the definitions are self-referential (if you will) -- they use the word being defined as a core part of the definition. Sometimes the definitions are almost ethereal. Of course being a long-time practitioner, I understand what is meant, but I think this will make it difficult for new practitioners and managers to grasp the definition. Here's an example from the definition of Dimension:

A general source of data that can be used to define various
types of segments or counts and represents a fundamental dimension of
visitor behavior or site dynamics.

What? IMHO, I should be able to use a standards document to develop an entirely new web analytics product, based on the standards. This doesn't tell me what a dimension really is, so I can't develop the product based on a standard.

To be sure, there's lots that's good about this document, and I don't mean to minimize that. But I would love to see this pushed more toward true standards and have some of the core definitions crystallized even further.

That's all for now. More later (maybe).

X Change Wrap Up

Eric Peterson wrote a very nice, big picture summary post regarding the X Change conference (thanks for the kudos on my huddle, Eric!). I have to agree with him, it was a very positive experience primarily because it was so interactive. I tend to get bored at conferences (both as presenter and listener) because what I crave is impassioned discussion, not a one way dissertation. No matter how much you know, there is always more to learn, even if you're the so-called expert in the room. X Change created the opportunity to have these impassioned discussions, and I, for one, took advantage of that. I learned a ton. And I think the people in my huddles learned a ton, too. Not from me, but from each other.

Here's Eric's closing statement:

I’ll leave you with this parting shot about X Change, a comparison I’m shocked that nobody smarter than I has already made:

• Emetrics is the Web 1.0 conference for web analytics where you will learn a ton and be very happy
• X Change is the Web 2.0 conference for web analytics where you will contribute a ton and be very satisfied

Well said.

X Change Day 1

Day one at X Change was pretty enlightening. I liked the huddle format, where leaders and participants were encouraged to engage in an open discussion on a topic, rather than fall into a presenter-listener relationship. Some huddle leaders were better at facilitating a discussion than others (some really just presented), but overall each of the huddles I participated in involved a good healthy dose of discussion and debate.

Of particular interest to me was the session on "deploying measurement systems across the global enterprise" with Judah Phillips. I came away from this session having synthesized this key idea:

In any global enterprise execution of a web analytics solution there are two frameworks (for lack of a better term) at work. Each framework contains multiple potential models. There's a framework for solution design models, and a framework for solution deployment models. The models in each framework can be mixed-and-matched -- there isn't a correlation necessarily between model 1 for design and model 1 for deploy. The combination of models that works for you will depend largely on the political and structural ecosystem you work in.

Here are the models:

Models for Design

1. Decentralized business units or entities with a unified measurement model
2. Decentralized business units of entities with a unique measurement model per bu
3. Centralized business with a single measurement model

The benefit of model 1 is that you gain the ability to roll-up business events across the globe and compare business units on an "apples to apples" basis because each unit is measured in the same way, reporting business events according to the same model of possible events.

The benefit of model two is that everybody gets what they want.

Model 3 probably only applies if your businesses around the world are essentially identical, and managed from one HQ location. Can't think of where else this would apply.

Models for Deployment

1. Crawl, Walk, Run (i.e. roll out basic analytics, then, as Judah put it, roll out dimensions that have meaningfulness to the business, then integrate external data)
2. Deploy "meaningful" solution slowly across globe

Here, the benefit of model 1 is that you can introduce people to the solution over time, and slowly raise their level of confidence and competence without overwhelming them.

Model 2, in my experience, is necessary when you have a decentralized organization that can not handle a quickly paced series of small changes, but instead offers you only one window per year (or less) to deploy a solution, or where the decentralized nature means that you have different windows at different times across the different parts of the enterprise around the world, preventing you from orchestrating a carefully controlled series of phases.

Of course, I think there are hybrids, too. I've worked with companies employing multiple combinations of the models for design and deploy...this is what I've seen. What am I missing? What other models are there?

More OLAP Fun

I've taken on a new project in the last few days. I'll be working to help an enterprise-class company integrate existing customer data into their Visitor Intelligence solution, allowing them to segment existing reports or build new reports, on the fly, with any combination of customer attributes from outside data stores and web analytics data from the page tag.

The power of the reporting and ad-hoc segmentation is as I wrote about here, but this is even more interesting because rather than segmenting and constructing reports only from data collected through page tagging, we'll be leveraging the power of web services and OLAP style reporting to integrate data about a single visitor from multiple databases and 3rd party systems.

I'll post more as the project moves forward.

Adding to Eric T. Peterson's Commentary on Dainow

So I've been keeping quiet on the Dainow post re: Google displacing all other web analytics "products". Partly because this has been fun to watch, but mostly because I work for one of the other supposedly "dead" competitors. I wanted to see where this landed before I got in the mix.

Eric Peterson's thoughts on this are spot on. Here's Eric:

Dainow demonstrates a near complete lack of understanding of web analytics and the web analytics marketplace. Google Analytics already dominates the market in terms of total domains coded, but dominance isn’t defined by the breadth of your coding, it’s defined by the success your customers have using your application!

I'll go one further. What Dainow fails to see is the difference between a product and a solution, where a solution is a product and a set of services combined to solve a business problem. While the market well served by Google doesn't really require a "solution" as much as they simply need a tool to do a job, the customers served by the big players (my employer included) tend to need (and have the money for) services to ensure successful solution design, implementation, deployment, and adoption of the tool set and the business processes required to make good use of the tool set.

The farther you go up the market, out of mid-market and into true enterprise class solutions, the more this is true. In fact, I would argue that in true enterprise-class solution deployments (the area of consulting I specialize in) the services are more important than the tool set. The greatest tool in the world isn't worth anything if it can't be successfully deployed across a global enterprise with a standards-based approach. Google, neat tool that it is, is nowhere near displacing the few vendors who can play at this level.

See you at X Change?

I'll be at Semphonic's X Change conference in Napa, California on Thursday and Friday. I'm leading two "huddle" sessions on using web behavioral data to optimize customer experience and drive business result improvements.

I'm looking forward to some lively discussion, and I'm hoping to learn as much as I share. I'll post a summary of the discussion points, ah-ha moments, and key take-aways from each session.

Hope to see you there!