Bloug: IA Heuristics for Search Systems

Sep 02, 2004: IA Heuristics for Search Systems

Another day, another project, another set of IA heuristics. A client asked me to kick the tires of their search system, so I decided to expand on the search aspects of the information architecture heuristics that we came up with a couple weeks back.

This time, I tried to align and categorize these guidelines with some common steps users take when searching a site. This semi-sequence goes like this:

Locating search: Where is it?
Scoping search: What will be searched?
Query entry: How can I search it?
Retrieval results: What did I find?
Query refinement: How can I search some more?
Interaction with other IA components: Can I switch to browsing when search isn't doing the trick?
Finishing search: What can I do now that I've done searching?

It might go without saying that these search heuristics are really geared to semi-structured text, not data; looking for ideas and concepts is a different undertaking than hunting for facts and figures (more on why they're different). Although there is definitely overlap, I'd love to see some references to data retrieval heuristics; if you know of any, or want to create your own set, please link to it in a comment below.

OK, without further ado...

Locating Search

Is the search interface located where you'd expect it to be? (Top of the page, and/or with other site-wide navigation options.)
Is the search interface always in that location? (It's always available, and always in the same place.)
Does the search interface behave consistently?

Scoping Search

Does it communicate what content is being searched?
Does it search what it should? (Users often assume that search covers all of an organization's content.)
Does it support search zones when appropriate? (Users may want the ability to search a specialized subset of content, such as products or a staff directory.)
Does the interface make it easy to select search zone options? (Long pull-down menus and overwhelming arrays of checkboxes are both quite common.)
Is contextual help with search zones available?

Query Entry

Is it obvious where in the search interface to enter a query? (Multiple search boxes can be confusing.)
Is it easy to enter a query? (Users can simply enter terms; more complex queries are addressed by showing examples or by allowing users to enter complex queries in syntaxes that may be familiar to them.)
Is the search box long enough to handle common query lengths? (Search log analysis can tell you how long most queries are.)
Are query builders used effectively (For example, spell-checking, stemming, concept searching, and thesaural searching; good choice of which builders are hidden, visible.)
Are syntaxes available that are appropriate for this site's users? (Certain groups of users may prefer to use wildcards, Boolean operators, proximity operators, etc.)
Are stopwords--semantically poor search terms (e.g., "the," "a," "her")--automatically removed from queries?
Is it obvious how to submit a query for processing? (The "search" or "submit" button is easy to find and is labeled clearly.)
Is contextual help with query entry available?

Retrieval Results

Is it clear what the query just entered was? (Most search engines can repeat the original query.)
Is it clear what was searched? (Especially important if your site employs search zones.)
Is it clear how many results were retrieved?
Can the number of results per page be configured by the user?
If search zones or federated search is employed, is it clear which zones results came from?
Are useful results available at the top of the list? (Wouldn't that be nice? Hard to test though.)
Do the most common queries produce useful results? (You can tell common queries from search log analysis; Best Bets are a great way to ensure useful results for those common queries.)
Are useful components displayed per result that help users select a relevant result? (These should help users understand enough about a result to distinguish it from others; known-item searches may benefit from displaying different and fewer components than open-ended searches.)
Are results listed in a useful way? (Sorting options might be quite helpful; listing each result's relevance score might be confusing.)
Are the results grouped in a useful way? (Usually results aren't grouped at all, but clustered results are becoming more and more common.)
Are categories for browsing displayed? (Yahoo is a great example of this practice.)
Are duplicate results removed? (Especially an issue with federated search.)
Is one result shown per document? (Often a document has been broken into separate files for display purposes; this floods result lists with duplicate results.)
Is contextual help available to help users understand how their results were determined?

Query Refinement

Is there a more powerful search interface available to help users refine their searches? (Often known as "Advanced Search," but my plea: use "Refine Search" or "Revise Search".)
Does this interface display the original search and make it easy to edit?
If a user chooses to narrow search results, are there easy and logical ways to do that? (The logical way to narrow may mean re-executing the query against only certain types of documents--"just technical specs please"), or searching within the current group of results; if so, is this easy to do?)
If a user chooses to expand search results, are there easy and logical ways to do that?
Is contextual help available to help users understand how to refine their query?

Interaction with Other IA Components

Is a search interface available when and where other IA components fail? (For example, allow users switch to search when they're about to give up on browsing.)
Does the search system take advantage of document tagging (i.e., metadata) to improve the relevance of results?
Does the search system take advantage of document tagging to improve the display of results? (Metadata can be used to display clustered results.)

Finishing Search

Can the user easily leave search and start browsing for relevant content?
Can a group of results be saved? (A group might be all of the results retrieved, or a subset that the user has selected.)
Can a group of results be emailed?
Can a query be saved for future use?
Can a saved query be designed to be executed on a regular basis? (This is useful for keeping up with dynamic content, like news.)
Is contextual help available to help users understand what they can do with the results they've retrieved?

Naturally, I'd love your comments/additions/deletions/etc.; I'm sure I've left out some obvious ones. Would you use a different set of categories than the sequential ones I've used here?

email this entry

Comment: Eric Scheid (Sep 2, 2004)

Great stuff!

We could also build up a bunch of questions which deal with how the site owner gets to manage their search tool.

1. Do they even have logs available to do log analysis with?

2. Do they keep a log of search-referrers (ie. where on the site did the user resort to search?) What about immediate-prior referrers (possible with a javascript trick)

3. Is their content date driven (eg. press releases), and does their search tool prioritise recent content over older content?

4. Is there a feedback loop between search queries and the spelling/theasaurus functions?

Comment: matt gregg (Sep 2, 2004)

the only thing I thought about that I didn't see was something along the lines of:
If no results are returned, does the system offer users ideas or options for improving their query based on identifiable problems with their input.
And then..is it easy for them to carry this search out.
This might be covered in some of the contextual help lines above, but those seemed to speak to having result to do something with.

a great checklist:)

Comment: Mike Jaixen (Sep 2, 2004)

Great list! My only complaint is that I wish I had this list about a month ago, when I was putting together some requirements for a search engine!

Comment: James Robertson (Sep 2, 2004)

Great list Lou, although it does strike me as targeting more expert users rather than general site visitors (narrowing and saving searches is pretty advanced).

This is another resource (that I had some involvement in writing) on search guidelines:

http://www.agimo.gov.au/practice/delivery/checklists/search

Comment: Lou (Sep 2, 2004)

James, thanks for the excellent pointer. I agree--some of these aren't appropriate for all users, just as some aren't for all sites (e.g., small sites wouldn't employ search zones). Maybe the best way to look at these is as a superset of what you'd actually use for a specific project.

Matt, another good point; it's somewhat implicit in a few of the guidelines listed above, but it's important to make this point crystal clear.

Thanks for the input everyone!

Comment: peterme (Sep 3, 2004)

Jeff Veen recently wrote an essay of interest to those who are seeking search heuristics:

8 Quick Ways to Fix Your Search Engine
http://www.adaptivepath.com/publications/essays/archives/000341.php

I do think you're missing some very basic stuff. Such as, "Do you treat '0 results' helpfully?", and "Do you handle empty queries well?" (Jared Spool had an interesting stat that among the most popular search queries is a blank search, the result of clicking on a "search" button, thinking it was a link to a search page, not realizing you were expected to enter text in a box to the left of the button.

Comment: Lou (Sep 3, 2004)

Good suggestions; another one is handling the entry of a URL into the search box, another common error.

Comment: Are Halland (Sep 3, 2004)

Me any my UX-team here at WM-data Norway recently published a report evaluating the "search experience" on 25 norwegian websites. Here's a short version of our list of evaluation criterias:

1. Finding, understanding and performing search: "Is it easy to find the search function, understand it and perform a search?". Sub-criterias included: Is the search field easily located and consistently placed throughout the website? Is the search field and search button presented intuilvely? Is it possible to perform a search without having to make selections?

2. Search results: "Are the search results precise, relevant and does the presentation make it easy to evaluate which results are relevant?" Sub-criterias included: Are the most important hits (as in answearing the question in the user scenario) presented first? Are there redundant,
outdated and/or trivial results? Are the result titles and descriptions relevant and well presented?

3. Navigation, filtering and "the way forward": "Is it easy to 'get beyond' the search result, i.e. by getting more or fewer hits, sorting results, etc?"

4. Combination of search criterias: "Does the search function understand what I mean when I use combinations of several search word, and does it give me a more precise result?"

5. Misspelllings and synonyms: "Does the search function understand what I mean even if misspell something or use an other word than the site owners?"

6. No hits: "Do I have start over when I get no hits, or do I get relevant help and tips to get to the results I want?"

As you can see, all of the evaluation criterias used can also be formulated as user questions.

The report is in norwegian only (sorry) but I did do a short english write-up of it, which can be found here:

http://www.brukaropplevingar.com/2004/09/search_experien.html

Comment: Eric Scheid (Sep 4, 2004)

is there suffienct detail in the result entries to determine relevance (eg. description, highlighted keyword hits, url, date of entry, date of last indexing, etc)?

Comment: Donna M. Fritzsche (Sep 4, 2004)

From a personal perspective this is great timing Lou! Thanks! It addresses some issues on a new project. I would also like to thank James Robertson for posting the link to the Australian Better Practices site. There is some very well-written material there (they cover a variety of topics in addition to search), and I found the examples to be immediately helpful.

Comment: Lou (Oct 5, 2004)

Thought I'd be remiss if I didn't mention Avi Rappoport's wonderful SearchTools site (http://searchtools.com/ ); any discussion of web searching should include a pointer to SearchTools.

Comment: BSalzman (Apr 27, 2005)

Great list, Lou, but you really should stop bagging on Advanced Search.

"Revise/Refine Your Search" cannot substitute for "Advanced Search" because, as the name implies, the user is revising or refining a search *that has already been performed,* whereas Advanced Search gives users advanced search options. Advanced search options and the ability to revise a search are two different things and not comparable.

Comment: Lou (Apr 27, 2005)

Perhaps, but users generally perform their initial query in a simple search box. So why expose "advanced" features from the start? Present them in a useful context, such as when that initial search didn't retrieve an especially useful result set.

That's why I like to bag on "advanced search". :-)

Comment: bsalzman (Apr 27, 2005)

Point taken. If a site allows advanced search options, I like to include a link to them up-front. Users familiar with advanced searching techniques appreciate this. It has been my experience that users who are not familiar -- or interested -- in advanced searching techniques are not offended by the word "Advanced."

Comments are now closed for this entry.

Comment spam has forced me to close comment functionality for older entries. However, if you have something vital to add concerning this entry (or its associated comments), please email your sage insights to me (lou [at] louisrosenfeld dot com). I'll make sure your comments are added to the conversation. Sorry for the inconvenience.