louisrosenfeld.com logotype

Home > Bloug Archive

Aug 18, 2005: Researching Search Log Analysis

Based on my recent posting, it might not come as a huge surprise that I'm co-authoring (with Rich Wiggins) a new book on search log analysis (SLA). I'm happy to report that we're already a couple chapters deep and I'm actually enjoying the process of writing, which usually requires a lot more self-discipline than my genetic programming supports.

I'm gung-ho on SLA because it seems so obvious, and yet it's still uncommon in the worlds of UCD and, more broadly, web design. Rich and I hope our book helps clear away many barriers to SLA--practical, technical, and political--by collecting both how-to info and justification in a single, short book.

Naturally, we hope we can draw on the wisdom of the masses. One starting point is a list of software tools that generate reports from search logs. Some are search engines, like MondoSoft; others are separate analytics tools, like WebTrends. (And of course, some of us roll our own.)

What commercial tool are you using to analyze search logs? If we can generate a reasonably comprehensive list of tools here, the next steps will be assembling samples of their reports and then comparing their features. And we'll be happy to cite helpful folks in the book's acknowledgements.

Onward, upward...

email this entry

Comment: Livia Labate (Aug 18, 2005)

Finally the cat's out of the box! ;) So, is the book going to discuss tools that do SLA and generate reports, or will it be about why/what to analyze and what types of conclusions can be drawn from certain types of analysis? Or both? or what? :)

Comment: Lou (Aug 18, 2005)

Yes. ;-)

We'll definitely discuss tools, but those change all the time. What, why, and mostly nuts'n'bolts *how* is what we'll emphasize, as well as case studies.

Comment: Margaret (Aug 18, 2005)

Nothing commercial per se, just Excel and some Visual Basic macros, but happy to share techniques for the equally under-resourced ;-)

If I have anything to do with the requirements list, then our new search engine, due in 2006, will (a) keep more sophisticated logs and (2) have built-in reporting tools.

Comment: Paul (Aug 19, 2005)

I use FunnelWeb for standard reports to go to commercial people on site performance. I use custom PERL scripts for user behaviour analysis.

Comment: George (Aug 19, 2005)

I use Web Log Expert for the usual stats and custom build modules for the .NET sites we are producing.

Comment: RTodd (Aug 19, 2005)

Search log analysis is an excellent topic and long over due. I am still amazed at how many individuals and corporations fail to review their log traffic and modify the content based on that behavior of the users. I have an academic paper being published this November (Proceedings of the 2nd International Conference on Knowledge Management.) on consumer based taxonomies which develop classifications based on the actual user activity captured in the search logs. Anyone interested in getting a copy, drop me a note.

Personally, my host company uses Awats which I think is an open source type utility component. At work, we use WebTrends and another emerging product from Microsoft. Perhaps a better question is how folks utilize the information versus which tool they use. More importantly, how will the utility of search change when every web site shares their actual usage data?

Comment: ML (Aug 22, 2005)

Tools was one of the last things I was begging for developers in my dept where I was before. I got some VB and Access stuff to help me get started. I ended up creating my own report just to help me study the activity and terms used. Open to sharing more personally.

Comment: Kit Seeborg (Sep 8, 2005)

WebTrends bought WebPosition and they just launched a new website for the WebPosition products. http://www.webposition.com
They've munged some WebTrends reporting into SEO analysis so it will be interesting to see how well that works.

Add a Comment:



URL (optional, but must include http://)

Required: Name, email, and comment.
Want to mention a linked URL? Include http:// before the address.
Want to include bold or italics? Sorry; just use *asterisks* instead.

DAYENU ); } else { // so comments are closed on this entry... print(<<< I_SAID_DAYENU
Comments are now closed for this entry.

Comment spam has forced me to close comment functionality for older entries. However, if you have something vital to add concerning this entry (or its associated comments), please email your sage insights to me (lou [at] louisrosenfeld dot com). I'll make sure your comments are added to the conversation. Sorry for the inconvenience.