louisrosenfeld.com logotype

Home > Bloug Archive

Apr 10, 2007: Analytics reports as organizational change agents

I'm wondering if anyone has experience using analytics reports as a means for achieving broad understanding, acceptance, and participation in an organization's internal knowledge management (KM) efforts?

We all know how difficult it is to get colleagues to participate in new initiatives—be they KM, information architecture, content management, or similar—especially when it means more work (e.g., tagging documents, following content authoring guidelines). Would disseminating regular analytics reports—with reporting on especially interesting and occasionally uncomfortable metrics—possibly serve as a tipping point to get employees interested enough to begin supporting and participating in these initiatives?

For example:

  • Would you be more likely to start tagging the documents you publish on your company's intranet if you had a chance of cracking this month's list of Top Ten Taggers?
  • Would you make a greater effort to follow corporate authoring guidelines if it meant your documents might show up on the latest list of Highest Rated Documents?
  • Would you find it interesting to see a list of your intranet's Most Frequent Search Queries with Zero Results on a regular basis? Would you try to author content to address these unmet needs?
  • Would you write better document headlines if it meant improving your documents' retrieval and usage levels?

Negative consequences might be interesting to explore as well. For example, would you author better document headlines if it meant not showing up on the company's Worst Titled Documents list?

I've already encountered examples where analytics thinking was quickly established within an organization because very basic reports were emailed to a few people who ultimately spread them virally. It's not surprising: numbers don't lie and they're often really interesting to at least a subset of any audience. Has anyone tried combining the viral aspects of emailing analytics reports with metrics that are perhaps more attention-grabbing—and personal—than what most analytics packages provide by default?

email this entry

Comment: Edward Vielmetti (Apr 10, 2007)


from The Onion:

Most E-Mailed' List Tearing New York Times' Newsroom Apart

April 7, 2007 | Issue 43•14
Include: Image Blurb
Most E-Mailed List Tearing New York Times Newsroom Apart
The Onion
'Most E-Mailed' List Tearing New York Times' Newsroom Apart

Copy and paste this code into a new post in Blogger, MySpace, or any other blog tool. It will display this Onion headline, picture, and teaser copy on your page, depending on what you select above.

It's up to you to write the rest of the blog post.

Text This Headline

Cell Phone Number:

From Email:

Powered by TeleFlip use to email to any cell phone

* E-mail
* Print
* blog this
* text this

Related Articles

* U.S. Dog Owners U.S. Dog Owners Fear Arrival Of Africanized Fleas March 23, 2005
* James Norton You Have Been Impregnated For National-Security Reasons June 2, 2004
* Miss Teen USA Seizes Crown, Declares Self Miss Teen USA For Life Miss Teen USA Seizes Crown, Declares Self Miss Teen USA For Life December 10, 1996

Email This Article
To: From:

NEW YORK—A feature on the New York Times' website that lists the stories most e-mailed by readers is destroying morale and escalating tensions among the once-dignified and professional Times staff, sources within the newspaper of record said Tuesday.

Comment: lynn in LA (Apr 11, 2007)

Hi Lou,
When I worked at a large software company, I created, and regularly issued a kind of searcher's hotlist, showing the delta on emerging query traffic, to great effect. At one point one of these reports, virally distributed, as you described, managed to unite a dozen different silos within the company all of whom were addressing different aspects of the same problem.

The report itself was extremely simple; just a few new queries, tracked over a finite period (usually a week), and the percentage increase or decrease. Often I would include a link to a relevant news article that might be driving some of these queries.

I don't know that it resulted in any great knowledge management outcomes, but it did get people talking and thinking together.

Back to grading... --Lynn

Comment: Aurelie Pols (Apr 11, 2007)

Hi Lou,

It actually really depends upon company culture and backing by management.

Of my experience, first of all, a lot of companies here in Europe are not always very enthusistic about sending data out in the open like that, even within the context of an Intranet. I've seen on many occasions data "taking a life of it's own", being wrongly interpreted and that usually backfires not to say that you might also be pinpointing that someone is actually doing a bad job.
So, once it gets personnal i.e. there is clearly someone to blame if they are on the Worst Title list, I wouldn't say it's a good idea.
On the other hand, if responsibility is blured too much, we all know this does not help action taking so it's really a very thin line to walk.

An intermediate way would be to point out to a small number of people the things that really need fixing and to prioritize. Once they have been fixed, show the underlying results to a larger group in order to not only show blame but also a path to ... enlightment?

But, again, it really depends upon company culture, something you get a feel for as the project is advancing.

Hope it help,
Kind regards from sunny Brussels,
Aurelie Pols
Web Analytics Team Leader
OX2 - WebAnalytics.be

Comment: Lou (Apr 11, 2007)

Aurelie, I like the path to enlightenment approach! (Enlightenment is always a good thing.) Pairing the data with actionable next steps is critical--I just don't know if it can be done within the context of the report itself very easily. At least not much in terms of specific actions. But the hope is that people who need help determining how to do a better job will have various resources (i.e., content authoring guidelines, or internal IAs) that will help guide them.

Lynn, what I'd pay to get my hands on a sample of your report...

Comment: Samantha (Apr 12, 2007)

I'll be starting to send targeted analytics reports to a variety of stakeholders next month with the re-design of the internal information-rich site that I manage, and one of my specific goals will be to help drive organizational change - specifically around ensuring that my organization (and processes) can be better constructed as necessary to increase the ability for people to find/re-find the information they need. We already know that our users look for a variety of information that our content contributors don't produce, but we haven't done a very good job of raising that information and potential solutions to those who can impact the make up and priorities of the organization.

I also think that these reports, if built for easy consumption such as including appropriate data visualization methods rather than just numbers and focusing on how the receiver wants to access this kind of information, will begin to drive home the fact that actual people who make business decisions come to our site. It is easy for our SMEs who provide the site's complex content to not realize how impactful their work can be to the broader organization because they are often emailing their work directly to the small number of people they directly work with. Driving home how much traffic from 'important' people the site gets, and inspiring a little friendly competition for that traffic should start to raise the importance of the site within my organization. This could really inspire some interesting organizational change. I'll keep you posted :)

Comment: Lou (Apr 12, 2007)

Samantha, thanks for sharing this.

So how are you determining who's an important user?

Comment: Samantha (Apr 12, 2007)

That is a great question. One goal of our SMEs is to provide research and insights that drive actionable business decisions. Of course, measuring how/when someone makes an 'actionable business decision' based on information received from a SME is a tricky problem. Sometimes it is learned about anecdotally, but we are trying to put in some processes that can help capture this without adding so much administrative overhead that no one will participate.

We do have a fairly good sense of which job roles (e.g. Product Planner) and which cost centers (e.g. Big Important Project cost center) we think comprise the most critical audiences who are making important business decisions. We plan to provide the analytics reports based on that information so we can target specific data to the variety of stakeholders depending on their role and area of ownership.

Add a Comment:



URL (optional, but must include http://)

Required: Name, email, and comment.
Want to mention a linked URL? Include http:// before the address.
Want to include bold or italics? Sorry; just use *asterisks* instead.

DAYENU ); } else { // so comments are closed on this entry... print(<<< I_SAID_DAYENU
Comments are now closed for this entry.

Comment spam has forced me to close comment functionality for older entries. However, if you have something vital to add concerning this entry (or its associated comments), please email your sage insights to me (lou [at] louisrosenfeld dot com). I'll make sure your comments are added to the conversation. Sorry for the inconvenience.