louisrosenfeld.com logotype

Home > Bloug Archive

Dec 05, 2005: Finding Users in Enterprise Environments

My pal Dean Karavite, until recently a usability specialist in an unspeakably huge enterprise environment, wrote to me about his challenges finding subjects for user studies.

Anyone familiar with enterprise environments knows how difficult it is to come up with a sample that's even remotely representative of the 14,127 audience segments typically encountered in enterprise settings. Some of the barriers are obvious--subjects are geographically distributed, they speak many languages, and there are just so many of them--but the biggest challenges are usually political. Often, you'll need the cooperation of product managers and business units to get in touch with the users they're closest to. But for reasons that range from logistical challenges and being hard to reach, to ignorance of the value of testing and even spite, your colleagues won't always help you contact sample users.

How did Dean deal with it? With the time-honored good old-fashioned "do it yourself" approach required for getting things done in enterprises: he built his own damn user pool:

Since I had five highly related development projects to manage, all applicable to a user population of 330,000 users and each with scores of user based activities planned for the year, I simply had to build my own user pool across the entire enterprise--all countries, business units and job roles with a boat load of demographic data on their IT needs, skills, adoption characteristics, support calls... and it worked out wonderfully.

I put out invitations on the corporate intranet and within a month or two I had 1500 people in the pool, all signed up to volunteer and prescreened with a demographic and IT needs/skills survey. This information was stored in a simple database, so when a project activity needed 20 users of such and such I could pull them right up. If they wanted business consultants from Asia, IT Architects from Europe and administrative assistants from Canada, I had them right there, off the shelf and ready to go. After we ran our first user activity I was able to completely satisfy every single interested party that they were represented in our selection of users.

Many people heard about my user pool and started asking if they could access "my" users for their work. This was impossible. However, it wasn't that I am incapable of sharing! 1500 candidate users may seem like a lot, but you can run through available candidates pretty quickly. Fortunately I followed my company's strict confidentiality rules which forbade me from passing out the list! (Heh heh heh.) Seriously, I was glad to help my peers set up their own user pools as an alternative. There were some simple guidelines to follow (many countries, especially in Europe, have strict laws about surveying employees). In addition, HR had the job role data to help guide selection and I tried to use standardized survey questions (available throughout the company) whenever I could.

Great work on Dean's part. He built his own pool rather than go through channels that were likely dead ends. Of course, Dean drew on existing infrastructure (e.g., help from HR, survey tools, and metadata available for classifying potential subjects) to create a sample pool for intranet users. And his subjects were somewhat self-selected, although I think this kind of biased sample is far better than no sample at all.

One of my clients, Caterpillar, is taking a more coordinated approach to building a user pool for its public web site. They've assembled a user advisory board that consists of people within the company who already maintain contacts with subsets of actual users, meaning they collectively have access to a large pool of test subjects. Much easier to find test subjects this way than looking for them ad hoc. The challenge is finding the right customer-facing people inside Caterpillar, but that's a far easier task than trying to identify subjects directly.

I'd love to hear about others' experiences with assembling user pools in large, distributed enterprise environments. Did you use Dean's grass roots "build it yourself" approach? Or has your organization taken a top-down, coordinated path toward maintaining a user pool, as Caterpillar is doing? Or something else?

email this entry

Comment: mantruc (Dec 5, 2005)

I'm currently facing the user pool challenge at a Bank we're working with.

We're about to have to migrate through different areas within the Bank because the one we first tried with was unable to find users willing to test.

We're taking a top-down approach currently, but we may end up having to do something creative...

Comment: Len (Dec 7, 2005)

I recently started a redesign project for a b2b website and I got help from the marketing managers.

On the first meeting I asked them to send me a list of participants from their unit that would fit the right profile to help me test the website.

But the tricky bit was to explain what kind of profile I needed and why I needed test participants. They just assumed that they were the only people that would "test" the site before launching.

I linked my explanation to the personas we created and asked them to look for people that had a similar position as the personas before they joined the company.

Comment: Alex Ford (Dec 19, 2005)

A very friendly site. Have a nice day! Beautiful Table is always Industrious Chips: http://www.sonypictures.com/homevideo/hellboy/ , Do Percieve Destroy - that is all that Gnome is capable of Loose Gnome is very good Table , when Opponents is Table it will Loose Cards to Create Chair you should be very Standard

Add a Comment:



URL (optional, but must include http://)

Required: Name, email, and comment.
Want to mention a linked URL? Include http:// before the address.
Want to include bold or italics? Sorry; just use *asterisks* instead.

DAYENU ); } else { // so comments are closed on this entry... print(<<< I_SAID_DAYENU
Comments are now closed for this entry.

Comment spam has forced me to close comment functionality for older entries. However, if you have something vital to add concerning this entry (or its associated comments), please email your sage insights to me (lou [at] louisrosenfeld dot com). I'll make sure your comments are added to the conversation. Sorry for the inconvenience.