Search Patterns

This article appears in slightly different form in the March/April 2010

issue of IEEE Micro © 2010 IEEE.

Designing for Discovery

Asking questions and wanting to know the answers is basic to human nature. No one knows of a time when this was not so. Researchers have speculated that human culture took a great leap forward when life expectancies increased to the point that children could interact with their grandparents. Word of mouth sufficed for a long time, but writing and printing led to further leaps forward.

It is a commonplace observation that sources of data are now increasing exponentially and that old methods of transmitting knowledge cannot keep up. Search technology has arisen to address this problem, but how best to use that technology remains an unsolved problem. Most computer users know how to go to Google or other search engines, enter search terms, and browse through results. Few user interface designers, however, know how to incorporate effective search technology into their designs. The book I look at this time addresses that problem.

Search Patterns: Design for Discovery by Peter Morville and Jeffery Callender (O'Reilly, Sebastopol CA, 2010, 192pp, ISBN 978-0-596-80227-1 www.oreilly.com, $39.99)

Peter Morville is a founder of the field of information architecture. He is author of "Information Architecture for the World Wide Web" and "Ambient Findability." His consulting clients include AT&T, Harvard, IBM, Microsoft, and the Library of Congress. Jeffrey Callender is the design director of Q LTD, a strategic design consultantly with clients worldwide. Peter bills himself as the word person. Jeff is the visual thinker. Their collaboration successfully integrates these ways of thinking.

Search, say the authors, is the worst usability problem on the web. We go to a website to find something -- a fact, a product, background information, an old acquaintance. Often, what we thought we came for is not actually what we need. Search is a conversation in which we discover new things and ask new questions. How quickly and accurately we find what we need defines the quality of our experience. If we give up and revert to more costly options like telephone or email, everyone loses. Yet that is what often happens when companies fail to use search technology effectively.

Patterns and interconnections

Apophenia, an aspect of madness and creativity, is the ability to see connections that others don't see. The savant Daniel Tammet sees patterns of shape and color in the digits of π that allow him to recite tens of thousands of digits accurately. To those who do not perceive such patterns, the feat seems impossible.

The authors are writing for an audience of highly sophisticated user interface designers who can hold their own with engineers and marketers and make the business case with upper management for investment in search technology. The authors' goal is to discover and study the patterns of search so that these designers can make search better.

The study of software design patterns owes its basic idea to Christopher Alexander, who wanted a language to help him describe the architecture of living spaces. In A Pattern Language, he recognized that how you put the patterns together makes all the difference. He analogized this to poetry. In prose we strive for simple sentences in which each word has an unambiguous meaning. In poetry the words come from the same language, but they have multiple meanings and connotations. They are packed densely together to express what prose cannot express efficiently.

Morville and Callender see poetry as the model for using the search patterns they present as the heart of their book. But they take the poetry model further. This is a densely packed book of interconnected meanings. Its 192 pages contain more useful information than many books of three times its length, but you must read slowly and carefully to extract that information. For example, here is how the authors identify the problem that they are trying to solve:

" . . . search is a wicked problem with no definitive formulation, considerable uncertainty, and complex interdependencies. Stakeholders have divergent goals and radically different world views. Requirements are incomplete, contradictory, and ever changing . . .

"Unlike Google, most firms aren't structured to manage the high-tech, step-changing, cross-functional, user-centered challenge. There are too many hyphens. As a hybrid of engineering, marketing, and design, search creates too many openings for missing links . . . Why risk your career (and your weekends) on a problem that's so intractable?

"We underfund and understaff search, and its poverty becomes a self-fulfilling prophecy. Users don't search now, because search failed then. Sometimes they browse. Often they bail."

To help describe patterns, the authors first break down search into its static anatomy and its behavioral aspects. The major elements of search are users, creators, content, the engine, and the interface. The many relationships among these elements help explain the complexity of search and the ideas in this book. For example, a search engine may take account of users' preferences, but if users don't bother to use the available customizations, they all wind up with the same default behavior. An interface may present results effectively, but if the searched content is full of redundant, outdated, trivial (ROT) material, users will not be able to find useful results quickly. Sometimes users are creators, when they give feedback about the relevance of results or comment upon a discovered item. The well known site that puts all of the elements together most effectively is amazon.com. The authors use Amazon as a good example of many patterns, but they point out that every situation is different. The Amazon approach works well because Amazon tailored its approach to the specific characteristics of its unique business.

The static anatomy helps us understand the elements of search, but to make specific situations better, designers must also understand the kind of conversation that leads to discovery in their environments. To succeed, search must have aspects of jazz. The process must flow. The authors cite Mihaly Csikszentmihalyi's contention that music, dancing, sailing, and chess are conducive to flow because they offer challenge, give control, support learning, reward skill, and provide feedback.

Just as designers repeat successful design patterns, users repeat the behavioral patterns that have worked for them. Search designers must support these behavior patterns. The main ways users interact iteratively with search facilities are narrowing/expanding searches and growing pearls. The main problems (antipatterns) they encounter are pogosticking and thrashing. To grow pearls, users find a good document and mine it for query terms and leads. Google supports this by offering links to similar items. Pogosticking is bouncing back and forth between a results page and the pages it points to. Thrashing occurs when users start from a flawed query (for example, using a misspelled term), then keep trying variations of it rather than switching to a better query.

Principles and patterns

The authors are interaction designers, and because they see little evidence of interaction design principles at work in search interfaces, they feel compelled to explain those principles. You'll have to read the book for the complete story, but here is a brief summary of some interaction design principles:

• Incremental construction lets users start small and build.
• Progressive disclosure hides complexity until it is needed.
• Direct manipulation provides visible objects for users to interact with.

The heart of the book is a discussion of ten design patterns. The choice of these specific patterns is meant as a convenient and somewhat arbitrary starting point. The authors describe each pattern, identify its most suitable contexts, and show how it relates to the other patterns. They discuss why the problem that the pattern addresses is common and why the pattern provides a good solution. They provide examples that you can visit, interact with, and study.

The Autocomplete pattern is the first and the easiest to understand. Typing is slow, especially on mobile devices. Users cannot spell, and they cannot remember search terms. As users enter the first few characters, the system begins to offer suggested search terms. This depends, of course, on having a good source of suggestions. Google does this well. Most sites do not.

The Best First pattern is deceptively simple. The system sorts results in such away that the first few results that users see are the ones most likely to be what they are looking for. A huge portion of what Google does behind the scenes supports this pattern. When implemented properly, a best first approach quickly gives users what they need or helps them formulate effective follow-up queries.

The Federated Search pattern addresses the problem of finding content that is distributed among a variety of content sources (silos). Implementing it entails problems such as slow performance and different query languages in different silos. Combining the silos into a single source may be a better approach to the problem. A good example of where to use federated search is hunting for books by author, title, subject, and so forth, across a set of independent university libraries.

The Pagination pattern emerged by accident, not as a response to a specific problem. Designers found it convenient to return a linear list of search results and to enable users to navigate through the list in screen-sized pages. Users soon discovered that pages provide a way to bookmark specific sets of results. Interfaces that replace the simple results page may have to provide a different approach to that use case.

The Faceted Navigation pattern gets away from the simple results page. It arises from the work of Marti Hearst and her Flamenco project at UC Berkeley. Using content and metadata from returned results, it constructs a custom visual map that makes it easy for users to narrow their queries incrementally. A drawback of this pattern is that it requires a large amount of screen space, so it does not work well on small mobile devices. The authors regard this pattern as the most significant search innovation of the past decade.

The Structured Results, Actionable Results, and Unified Discovery patterns all contribute to speeding discovery and making the process more iterative and interactive.

Moving search forward

Practitioners will especially appreciate the chapter called Engines of Discovery. Having laid out the principles of interaction design and created a language for discussing search patterns, the authors examine and discuss a number of sites that explore new ways to help their users discover what they may not have known they were looking for. Visiting these sites and understanding how they use search patterns is key to deriving maximum benefit from this book.

The authors hope that this book will influence the future of search technology. They see themselves not just as technologists and communicators, but as persuaders. In support of that function they have created some futuristic scenarios that they hope will serve as mythic guides for search technologists. These incorporate neural implants, direct communication of raw emotion, artificial intelligence, rogue agents, and software to recognize and track people.

This book is a monumental achievement. The authors have provided a guidebook to a largely unexplored and immensely important territory. Reading it requires attention, effort, and exploration. Not reading it is likely to lead you out to pasture.