Measuring the Right Things [part 2] - Defining your KPIs

I've been thinking about this post for a few days now, trying to figure out just the right way to approach it. What's got me thinking about it is that I'm getting ready to engage with a new client to help them define a Key Performance Indicator framework they can use to enforce a more disciplined approach to site management within the organization. Additionally, they want a framework with which they can document the business value of the site to executive management.

Conceptually, defining KPIs is extremely simple...

1) Understand your Root Goal or Overarching Business Objective
2) Document the efforts you'll undertake or approaches you'll use to try to achieve those goals
3) Document the desired visitor behavior to result from your efforts
4) Express the ratio or data-point that lends insight into the scrutinized visitor behavior (this is a KPI)
5) State the individual measures that create the ratio or provided needed context to understand the KPI

What end up on your scorecard or dashboard is nos. 4 and 5. Numbers 1 through 3 you use to provide the overall business context when you write up your analysis of overall website performance for management.

Where most client-side analysts or site managers tend to fall down when deciding what to measure is failing to explicitly state and understand the Root Goal that 'efforts' and 'behaviors' are derivative of. Most of the time when I ask a client what their goals are they tell me about desired behaviors or eff0rts to affect behaviors, but they don't talk about the high-order objectives -- the Root Goal. As is true with any business, you need to understand your goals before you can understand your performance, or even what you need to measure to understand performance.

Here's an example of a fully defined KPI for a retail banking site:

1) Root Goal: Grow new account revenue 20%
2) Effort (or Approach or Method): Improve attach rate (cross sell performance)
3) Desired Visitor Behavior: Complete application with accessory products
4) KPI: % applications with accessory products
5) Contextual Data: # of applications, # of applications with accessories, AVG Revenue/Application (without attached accessory), AVG Revenue/Application (with attached accessory)

A colleague forwarded me this post from Craig Schiff while I was working on this post. In it, Craig does two things nicely (aside from being a smart guy): he provides a nice graphic illustrating the top-down definition of KPIs; and he clearly demonstrated to me how web analytics is just a little slice of BPM (Business Performance Management). We're working on exactly the same challenges, just on a different scale. The wording in his process is slightly different from mine, but they both get you to the same place.

Here's Craig's illustration:


One other note about Schiff's process -- I liked step 5 so much I included it in mine. I had not explicitly called out the needed contextual data separately. Instead, I was just including the contextual data as part of Step 4. It's much clearer when you call those out separately.