Oct 05, 2004: Ignoring Information Needs
A colleague is working with a user group of seniors, and had found some interesting research on how they interact with an information architecture. She wanted my take on it; here's her summary:
Michael Lin conducted a study (263 Kb PDF) with older adults that was published in 2003 in Computers in Human Behavior. To test his hypothesis that older users would be better oriented in the hierarchical information architecture than in the network information architecture, Lin measured
- the number of "nodes opened" (pages visited)
- the number of nodes opened repeatedly
- the number of "additional links searched"
- time spent in finding the answers to the task scenarios.
Participants using the hierarchical information architecture opened more nodes and spent significantly more time. However, the number of nodes repeatedly opened was much lower than for users of the network information architecture.
The author suggests that the result related to repeatedly opening nodes is more important. He concludes that opening more nodes repeatedly in the network information architecture shows that those participants became disoriented. Based on this finding, he suggests that the hierarchical information architecture is best.
I'm not entirely comfortable with this conclusion. The experiment may have been flawlessly executed, but I'm not sure that it asked the right question. The tasks that were tested didn't necessarily mirror the kinds of information needs that users might have actually had; instead, the experiment's focus was on how a specific age group interacted with information systems that had different shapes (hierarchy versus hypertext).
Certainly age and other demographics are critical to determining how users might interact with information. But I like to think that there's more to users than how a census bureau might characterize them. Demographic views make it easy to segment audiences for determining testing samples. However, demographics don't necessarily have much to do with how people actually use information.
As an alternative, I suggest focusing on testing tasks based on users' most common information needs. Information needs are difficult to determine, but have quite a lot to say about how our designs should support users at the very moment they're using our systems. For example, if known-item needs predominate, then perhaps the site's architecture should have an especially strong search system; if users are often exploring a subject, then a site map or other form of navigation might be worth investing in. it just seems to make sense that our behavior would be at least as likely to be influenced by the type of information we seek, such as an answer to a specific question or results that help us learn more about a topic, than by our age or gender.
Others share this concern. Donna Maurer recently noted that we often tell subjects what to find, and often this is a known item, the easiest information need to fulfill and to test. Donna writes: "A very, very large proportion of our body of knowledge about how people approach sites, and about how we should design sites, is based on a very narrow activity of looking for known information. And in most cases finding the information is seen as the end result." Donna worries that this approach is now so firmly encoded into our testing methods that our subsequent designs may require serious reexamination. "We may have already gone a significant way along the wrong path, where findability is king, where we spend more time on designing navigation than on designing answers and where we may be missing a major part of information seeking."
I'm trotting out onto thin ice here, as I don't have the science to back up my concerns. There may be supporting studies in the LIS and IR literature (and I'll bet Marcia Bates' work would be a good place to look). I'm simply blogging my opinion here, and fortunately for me, opinions don't require a lit search. As an opinionated practitioner, I would love to see IA move into academia and push forward a research agenda that asks the tougher questions and provides us practitioners with more actionable, practical results.
Andrew Dillon, get cracking!
email this entry
Lou (Oct 5, 2004)
Just surfing for info on search metrics, and found a quote from Delphi over at the CIO site: "Hadley Reynolds, vice president and research director at Delphi Group, says it often helps if users know exactly what information they need. The problems arise, he says, when a user wants to browse and discover something new." (http://www2.cio.com/metrics/2004/metric692.html )
Donna Maurer (Oct 5, 2004)
I'm going to do my masters research project on this very issue and am this week writing up my lit review.
A large proportion of the academic research into hypertext is done this way - tell people what to look for and count how long it takes them to find it, how many nodes they open, how 'lost' or disoriented they get. See if they can draw a map of the structure.
The focus is strange - it is all about 'navigating' to information objects with the 'finding' of the right information being the goal. There is little focus on whether 'navigating' actually helps people understand the information, how people know that they have found the 'right' information and what makes the information suitable for a task.
Andrew's research into shape, and other genre work at least examines the real goal of using information for a task or decision and looks at how people understand this information. This is infinitely more useful than another study on which navigation method works best ;)
Unfortunately for me, I do require a lit search, and based on my research, your opinion is spot on!
// so comments are closed on this entry...
©2008 Louis Rosenfeld LLC. All rights reserved.