Being Geek

This article appears in slightly different form in the November/December 2010 issue of IEEE Micro © 2010 IEEE.

This time I look at a book that is remarkable for its author's exceptional insights and uncommon points of view.

Being Geek: The Software Developer's Career Handbook by Michael Lopp (O'Reilly, Sebastopol CA, 2010, 334pp, ISBN 978-0-596-15540-7, www.oreilly.com, $24.99)

Michael Lopp has worked in Silicon Valley for more than 15 years. He is an engineering manager at Palantir and has worked for ground-breaking companies like Netscape, Apple, Borland, and Symantec. He was also part of a failed startup. He is the author of Managing Humans and the Rands in Repose blog. Being Geek draws heavily on postings from that blog.

Geeks

Lopp's target audience consists of software developers and people like them. He calls them nerds or geeks interchangeably. He chose the word geek for this book because he liked the sound of the resulting title. 

If you're one of Lopp's geeks, you're socially handicapped in ways that he describes in a chapter called The Nerd Handbook, a guide to give to your friends and family. Your sense of humor grows out of your bitterness over your early life as an outcast. You have a project, and it's always on your mind. Your relevance detector tunes out small talk, making it difficult for you to relate to others. You are a systems thinker. Your model of the world is a computer. If you know the rules, you can figure out what to do next. Ambiguity makes you anxious.

Unlike many software developers, Lopp has no difficulty interpreting the non-verbal aspects of his surroundings. He observes the behavior of colleagues and generalizes the lessons he learns. He picks up on the unspoken cues in situations, so developments rarely take him by surprise. Following some of Lopp's advice, however, may require more facility at interpreting the non-verbal than many potential readers have.

Rules of three

Lopp thinks strategically, so he can react to unpleasant surprises by remembering the big picture. He may have a manager in his job, but Lopp is the manager of his own career. His two-pronged approach to this book is as follows:

• Teach you a system of improvisation so that you can navigate the day-to-day unpredictability of the work world.
• Help you develop a strategic career plan, so you know what to do if the sky falls in your current job.

To teach you a system of improvisation, Lopp takes you through the kinds of situations that are likely to arise in software development firms and helps you understand the underlying system and rules. He shows you tools and techniques for managing your time and workload. 

To help you develop a strategic plan, Lopp provides a narrative—a single job in all its aspects, from interviewing to leaving, seen largely from the perspective of a manager. He likes the number three, so he gives you three questions and asks you to keep them in mind as you follow the book's narrative: 

• What am I doing? 
• What do I do? 
• What do I care about? 

To Lopp, your day-to-day work, your management philosophy, and your career plan should be driven by another set of three questions:

• Am I setting the technical direction?
• Do I know what I have to do to grow?
• Am I keeping my commitments?

Lopp uses technical direction as a shorthand for all of the technical aspects of your career, from choosing an area to work in to writing a piece of code. You might not get to make all of the decisions, but if you write code, you should know what’s best for it at all stages of its life.

Failure to grow brings quick death in the technical world. Information has a short half life. If you haven’t failed recently, if nobody is challenging you, if you haven’t learned anything significant this week, you are in danger of falling behind. 

Failing to keep a commitment, even a minor one, puts a scar on your reputation that is hard to erase. Your reputation is a huge asset that you should maintain at all costs. Sometimes this means telling your boss that you can’t handle that one extra task. Better to say no than to say yes and not do it.

There, in a nutshell, is Lopp's philosophy. But the best thing about the book is Lopp's insight into situations, management techniques, and tools. The bulk of the book explores all aspects of a hypothetical job. 

Job hunting

Lopp starts with the decision to leave your current job. After three years in a job—three releases for the product to become real—you might start to feel the itch to leave. Don’t mistake anger for the itch—cool off first before you act. When you do feel the itch, however, it’s time to go. You might have reservations about whether you’re really done or about the people you’re leaving behind, but Lopp has good answers to those reservations. You’re never really done, and your network transcends individual companies. Start looking for your next challenge. 

Many books contain advice about resumes, portfolios, and interviews. This one is as good as any. Most of it is in line with other books, but Lopp addresses one area I haven't seen elsewhere. As an interviewee, you want to help the interviewers get what they need, but you need to elicit information too. Lopp talks about finding the "button" that will get each kind of interviewer talking. Some interviewers arrive angry, some love to talk, some try to maintain control, but for most of them, there is a strategy that will loosen their tongues.

Lopp also looks at some of the dangers that can catch a hiring manager napping. Requisitions look good on paper, but they can disappear in an instant. He recommends spending at least an hour a day on each open requisition. He also advises laying some groundwork before you have a requisition. Build a network of people you can call when the requisition appears.

Sometimes someone you thought you had hired decides to stay where they are. Lopp believes that this is often because you didn’t let them know how much you want them. He advises keeping in touch from the day they accept the offer until the day they show up for work.

People on the job

Lopp emphasizes the kinds of informal interactions and relationships that don't show up on the organizational chart. When he was at Netscape, for example, four people who didn't look especially important on the organizational chart met each day to play bridge. He soon learned that they set much of the technical direction and corporate culture. When one of them left the company, long before the visible signs of trouble, Lopp knew that it was time to go.

Another interesting bit of analysis that Lopp performs concerns what happens when you make a big mistake. Your boss temporarily assumes a new persona: Interrogator, Illuminator, Randomizer, Prioritizer, or worst of all, Enemy. The temporary change may reveal attitudes and communication problems you didn't know existed. Lopp characterizes each of these reactions and shows how to restore your boss's usual calm, supportive personality.

Lopp brings his insights to other common situations. Your group gets sudden bad news and a number of personas appear, some requiring careful treatment. One displays explosive anger. Another refuses to accept the news. Yet another sees the sky falling. One blames himself while another says, "I quit." Lopp shows the course that each of these reactions is likely to take and explains how to get everyone into problem-solving mode.

Interpersonal relations take work, but some people are too much work. Every group has its values and beliefs. Those who are fundamentally incompatible with those values and beliefs must go, no matter how good they are. As Lopp puts it, the history of Silicon Valley is full of toxic people who were right. Their agenda and ideas were right for their company. Nonetheless, they had to go.

Lopp believes in the Pond. The workplace supports many information flows—official and unofficial. If you're there, you pick up on them. If you're not, you don't. When someone on his team wants to work remotely, Lopp may feel he has to say yes, but he worries. Stepping out of the Pond may be that person's first step toward leaving. On the other hand, the modern world makes it difficult to avoid remote workers altogether. Lopp identifies the factors in the employee and in the group that are essential to making it work. 

Games

Lopp has two important roles for games in his management approach: stimulating activity and developing relationships. In one example, he uses a list of bugs on a whiteboard and a system of scoring for analyzing and fixing them. Players use different colors to record what they do, and everyone can see who is accumulating points.

Back Alley Bridge, known by its acronym, BAB, has been around for at least fifty years. It looks a bit like Bridge and a bit like Hearts. The bidding is a bit like Pinochle. An important aspect is the trash talk that accompanies play. Lopp uses it to build trust between two managers who aren't getting along.

The Werewolf game has made the rounds of the Internet in various forms. Each night in the village, the werewolves kill someone. Then the remaining players awake and try to identify a werewolf to kill. They have nothing to go on but what they observe, as one after another is accused and presents a defense. Lopp believes that playing this game will sharpen your interpersonal skills and prepare you for business meetings.

Networking

When people talk about networking, they usually mean building a network of contacts throughout the industry. Lopp has a slightly different emphasis. He tells you to find "your people." Growing up, you may have been an isolated one in a thousand. But now you can improve your odds considerably. Go to conferences. Join affinity groups. Contribute to mailing lists. You will know your people when you see them. Writing about what you care about can be your initial connection, but extended face-to-face interactions show you what email hides: I talk with my hands. You can't stand the distraction of looking anyone in the eye when you're thinking hard. 

Managing time

Lopp has a lightweight system for tracking the tasks he should be working on. His artifacts are a to-do list, a parking lot list, and a trickle list. He reviews them briefly twice per day—the morning scrub and the evening scrub. His underlying principles: you won't complete everything you should; if you drop an important task, it will find its way back. 

The morning scrub starts with yesterday's to-do list. If you intend to do an item today, leave it on. Don't assign it a priority or a completion date. If you intend to do it after today, use it to start today's parking lot. If you don't have a realistic plan for doing it soon, throw it away. The evening scrub is similar. 

The trickle list tracks long-term goals. The column heads are types of activities, like read a book or have a hallway conversation. Each day gets a row of Xs and Os. Looking at the list can remind you of things you need to do or changes you need to make to your long term priorities.

Lopp has a framework for deciding which tasks to take on. Some things that come his way are crises. He has to take them on. In addition, he insists on devoting some time to creative activities—sitting in a room and thinking or doodling on the whiteboard. He hopes to delegate everything that falls between—routine matters—to someone else.

Bits, features, truth

Everyone talks about the triangle of time, quality, and features. You can have any two of the three. If you think about this metaphor for any length of time, it falls apart. Lopp replaces it with the idea that on any project there should be three separate authorities, one each for bits, features, and truth. These are the traditional roles of the engineering manager, the product manager, and the program manager, but thinking in terms of the actual functions may be more realistic.

The people occupying the three roles should be equally strong. There should be both a healthy tension and mutual respect between any two roles.

A complaint

My main complaint is about editing. The credits list an editor, a production editor, a copyeditor, and a proofreader. Nonetheless, the book has many typographical errors. The writing style is far from crisp. Furthermore, I feel that it suffers from a serious error of editorial judgment. The writing is full of unnecessary profanity. Even if the author speaks that way, there is no need to reproduce the vulgarities in print. They add nothing essential.

I recommend this book highly because of its content. This review covers many aspects of the book, but there are important topics I don't mention. If you have anything to do with software development, or if you just feel geeky, I hope you'll buy the book and study it.

Miscellany

This article appears in slightly different form in the July/August 2010

issue of IEEE Micro © 2010 IEEE.

My backlog of books to review has grown out of hand. I don't have time or space to do justice to all of them. So here are some interesting books with brief explanations of why I hope you'll find reading them worthwhile. I recommend all of them.

Math and philosophy

Recently I've read two books co-authored by Ellen Kaplan. She wrote one with her husband Robert and one with her son Michael. Both books are simultaneously erudite and readable, with delightful allusions to many aspects of culture. The books deal with some of the philosophical and technical underpinnings of the modern world. 

Another math book, by a physicist, focuses on how to solve thorny problems using heuristic techniques, approximations, and lucky guesses.

The Art of the Infinite by Robert Kaplan and Ellen Kaplan (Oxford University Press, Oxford, 2003, 334pp, ISBN  978-0-19-517606-3, www.oup.com, paper, $16.95)

In 1994 Robert and Ellen Kaplan founded The Math Circle (www.themathcircle.org), a school "designed for students who enjoy math and want the added challenge of exciting topics that are normally outside the school curriculum." In this book they introduce us to numbers, plane geometry, and some of the mathematicians who have furthered the study of these subjects. Because some of the deepest results of mathematics arise from considering the infinite, the authors try to ensure that we understand the definitions and philosophical underpinnings of that concept.

Georg Cantor's proof that the infinity of real numbers is greater than the infinity of whole numbers is well known and easy to understand. The authors show us the painful years-long path Cantor followed to arrive at that insight. This kind of humanization of mathematicians is one of the book's strengths. The intuitionists and the formalists, like Swift's big-endians and little-endians, both turn out to have the same sorts of foibles as the rest of us.

Chances Are: Adventures in Probability by Michael Kaplan and Ellen Kaplan (Viking, NY, 2007, 332pp, ISBN 978-0-1430-3834-4, www.penguin.com, $26.95)

The dust jacket of this book bears a photograph, taken by Robert Kaplan in 1961, showing Ellen Kaplan pushing Michael Kaplan in a stroller. Subsequently, Michael studied European history at Harvard and Oxford. He is now a writer and film maker. Ellen is a classical archaeologist. She has taught math, biology, Greek, Latin, and history. Neither author is a professional mathematician. Perhaps their backgrounds explain their ability to see the big picture.

Probability and statistics have infiltrated practically every important aspect of our lives. It began simply enough when gamblers sought help from mathematicians like Fermat and Pascal. The resulting mathematics soon helped sea traders hedge against the unpredictable risks their ships faced with each voyage. The insurance industry expanded to cover other commercial and personal uncertainties.

Insurance and medicine have become entwined. While death panels may be a fantasy, medical practice and reasoning are firmly grounded in probability. In 1976 I wrote a review of Galen & Gambino's Beyond Normality: The Predictive Value and Efficiency of Medical Diagnoses (Wiley, 1975). Their work explained how to apply Bayes' theorem to evaluate the value of diagnostic testing. At the time most physicians were only beginning to understand this type of reasoning. Now it controls much of what they do.

Legal reasoning is another area where thinking about probability can be useful. It does, however, sometimes lead to highly unintuitive results. This has led appeals courts to reject expert testimony based on Bayes' theorem. According to the authors, "the technique has been forbidden, not because it doesn't work, but because the court may not understand it. We can continue to err, as long as we err in ways we find familiar."

Game theory and military planning underlie decisions that affect our very existence. In fact, the Cold War came to a peaceful end largely because leaders cast aside the "best" strategies in favor of a more humane view of human behavior.

I don't think you can understand today's world without understanding the concepts in this book. The authors have put it all in one place for you. Get it and read it.

Street-Fighting Mathematics: The Art of Educated Guessing and Opportunistic Problem Solving by Sanjoy Mahajan (MIT, Cambridge MA, 2010, 150pp, ISBN 978-0-262-51429-3, mitpress.mit.edu, 25.00)

Sanjoy Mahajan, a physicist, is the associate director for teaching initiatives at the MIT Teaching and Learning Laboratory (TLL). The TLL works with faculty, teaching assistants, and students to promote excellence in teaching and learning.  

Carver Mead, one of Mahajan's thesis advisors at Cal Tech, says in the foreword to this book 

Most of us took mathematics courses from mathematicians -- Bad Idea! Mathematicians see mathematics as an area of study in its own right. The rest of us use mathematics as a precise language for expressing relationships between quantities in the real world, and as a tool for deriving quantitative conclusions from those relationships.

So if you want the answer, or something close to the answer, and you don't care about the proof, the techniques in this book might be just what you're looking for.

Mahajan considers his work to follow the path of the great mathematics teacher George Polya. Polya cited Euler and Laplace as mathematicians who perceived that "the role of inductive evidence in mathematical investigation is similar to its role in physical research." Some of the techniques that Mahajan describes are similar to the ones Euler used to find the sums of infinite series in the 1700s.

While the ideas Mahajan puts forth are relatively simple, many of his examples would be difficult to follow for someone without at least the prerequisites for an MIT education. If you have those prerequisites, you might enjoy reading this book.

Grammar, usage, and editing

Ambrose Bierce's Write It Right: The Celebrated Cynic's Language Peeves Deciphered, Appraised, and Annotated for 21st-Century Readers by Jan Freeman (Walker, NY, 2009, 236pp, ISBN 978-0-8027-1768-9, www.walkerbooks.com, $24.00)

Just over a hundred years ago, Ambrose Bierce, a journalist better known for his cynical Devil's Dictionary, wrote Write It Right, a quirky collection of prescriptions for writers. Recently, Jan Freeman, who writes a language column for the Boston Globe, decided to examine Bierce's maxims. She apparently decided to do so for an indirect reason. Bierce's fiercely expressed complaints are largely irrelevant today. They have been replaced by new pet peeves. By showing the irrelevance of most of Bierce's complaints, she helps us gain perspective on our own hobgoblins. They may be just as quaint a hundred years from now as Bierce's are today.

Bierce, like many language purists today, based many of his rules on a belief that English should be logical. But many of the illogical usages he complained about had been established hundreds of years before, and many persist today. Freeman makes this clear again and again as she carefully analyzes all 441 of Bierce's rules. If you're interested in usage issues, you'll enjoy this book.

Making Word Work for You: An Editor's Introduction to the Tool of the Trade by Hilary Powers (Editorial Freelancers Association, NY, 2009, 80pp, ISBN 978-1-880407-22-6, www.the-efa.org, spiral, $15.25)

Now in her fourth career, Hilary Powers has been a freelance editor since 1994. She says she chose editing to enable her to emulate Nero Wolfe, that is, never to have to leave home on business. Before her first year as an editor was done, she had abandoned paper. She works only online, a fact that has necessitated her mastery of Microsoft Word.

The target audience of this small, spiral-bound book is anyone who edits for a living. Anyone who uses Word, however, will find useful information in it. For example, editors must master Word's change-tracking facilities, but many non-editors use that feature too. Macro programming is another powerful feature of Word that will quickly repay your learning how to use it well. Powers shows you where and how to use macros and provides free downloads of her own macros to help you learn the details.

Whether the task is mastering macros or simply selecting configuration options, this book removes your excuse for avoiding it. If you're like most Word users, you could do a lot of what this book recommends without reading it. Once you have this book, however, and read about the huge increases in efficiency that Powers achieves, you won't be able to resist the urge to tinker.

I have known Hilary Powers for many years, so I can testify that she writes the way she speaks. Her style is clear and colorful, never boring. The topics she covers are practical and directly useful. If you use Word, you should read this book.

Communicating

Confessions of a Public Speaker by Scott Berkun (O'Reilly, Sebastopol CA, 2010, 240pp, ISBN 978-0-596-80199-1, www.oreilly.com, $24.99)

Scott Berkun is a technical author who decided to try to make a living as a public speaker. He succeeded, and this book tells how he did it and what he learned along the way. None of it is glamorous -- even his being interviewed with prominent business leaders by Maria Bartiromo on CNBC. Berkun shows you the tricks of the trade -- the drab details and shameless gimmicks that help ensure that your audience goes away happy.

One of the parts I like best is "The Clutch Is Your Friend." This chapter title comes from something Berkun's brother told him to help him learn to drive a car with a manual transmission. It introduces some excellent rules about how to teach -- whether you're facing one person or 5000.

Berkun says, "Everything in this book is true and written to be useful, but if you don't always want to hear the truth, this book might not be for you." By this he means that after seeing how sausage is made, you might never want to eat it again. That hasn't been my experience. I find the material fascinating. If you ever speak in public, you'll learn something useful from this book. 

The Back of the Napkin: Solving Problems and Selling Ideas with Pictures, expanded edition, by Dan Roam (Penguin, NY, 2009, 300pp, ISBN 978-1-59184-306-1, www.penguin.com, $28.95)

Dan Roam designs presentations and gives seminars about visual thinking. Here is his elevator pitch for this book: 

"Visual thinking means taking advantage of our innate ability to see -- both with our eyes and with our mind's eye -- in order to discover ideas that are otherwise invisible, develop those ideas quickly and intuitively, and then share those ideas with other people in a way they simply 'get.'"

Those of us who communicate for a living know that visual communication is an important tool, but few of us feel comfortable using that tool. The biggest obstacle that Roam must overcome is I can't draw. He allays that concern in this book by using simple drawings made from rough hand-sketched elements. Nobody draws worse than I do, but I can replicate any of the drawings in this book -- at least well enough to communicate effectively.

In planning how to explore or express an idea visually, Roam asks five questions about what he wants his drawing to emphasize:

 * Simple or elaborate?

 * Qualitative or quantitative?

 * Vision or execution?

 * Individual or comparison?

 * Change (delta) or status quo?

This gives him the colorful mnemonic SQUID, for which he provides a minimal but recognizably squidlike sketch. 

Roam applies the SQUID questions in each of his main areas: discovering, developing, and selling ideas. On this framework he hangs profusely illustrated examples. All you have to do is read along with pencil and paper at hand, and you'll learn that you really can use visual thinking effectively.

Conversation and Community: The Social Web for Documentation by Anne Gentle (XML Press, Fort Collins CO, 2009, 236pp, ISBN 978-0-9822191-1-9, xmlpress.net, paper, $29.95)

Anne Gentle is a technical writer who ran a company blog in the prehistoric days of that technology (2005). She learned about using wikis for documentation by volunteering for the One Laptop per Child project. She also works with FLOSS Manuals, a community that supports tools for producing free documentation for free software.

Gentle sees a structural shift from one-way product documentation toward interactive, collaborative product support that includes plenty of user-generated content. Technical communicators steeped in the old model need help understanding and moving to the new one. This book provides such a guide. Gentle proceeds methodically through the new technologies and discusses the issues that arise for communicators.

If you have a role in technical product documentation, you should read this book.

Search Patterns

This article appears in slightly different form in the March/April 2010

issue of IEEE Micro © 2010 IEEE.

Designing for Discovery

Asking questions and wanting to know the answers is basic to human nature. No one knows of a time when this was not so. Researchers have speculated that human culture took a great leap forward when life expectancies increased to the point that children could interact with their grandparents. Word of mouth sufficed for a long time, but writing and printing led to further leaps forward.

It is a commonplace observation that sources of data are now increasing exponentially and that old methods of transmitting knowledge cannot keep up. Search technology has arisen to address this problem, but how best to use that technology remains an unsolved problem. Most computer users know how to go to Google or other search engines, enter search terms, and browse through results. Few user interface designers, however, know how to incorporate effective search technology into their designs. The book I look at this time addresses that problem.

Search Patterns: Design for Discovery by Peter Morville and Jeffery Callender (O'Reilly, Sebastopol CA, 2010, 192pp, ISBN 978-0-596-80227-1 www.oreilly.com, $39.99)

Peter Morville is a founder of the field of information architecture. He is author of "Information Architecture for the World Wide Web" and "Ambient Findability." His consulting clients include AT&T, Harvard, IBM, Microsoft, and the Library of Congress. Jeffrey Callender is the design director of Q LTD, a strategic design consultantly with clients worldwide. Peter bills himself as the word person. Jeff is the visual thinker. Their collaboration successfully integrates these ways of thinking.

Search, say the authors, is the worst usability problem on the web. We go to a website to find something -- a fact, a product, background information, an old acquaintance. Often, what we thought we came for is not actually what we need. Search is a conversation in which we discover new things and ask new questions. How quickly and accurately we find what we need defines the quality of our experience. If we give up and revert to more costly options like telephone or email, everyone loses. Yet that is what often happens when companies fail to use search technology effectively.

Patterns and interconnections

Apophenia, an aspect of madness and creativity, is the ability to see connections that others don't see. The savant Daniel Tammet sees patterns of shape and color in the digits of π that allow him to recite tens of thousands of digits accurately. To those who do not perceive such patterns, the feat seems impossible.

The authors are writing for an audience of highly sophisticated user interface designers who can hold their own with engineers and marketers and make the business case with upper management for investment in search technology. The authors' goal is to discover and study the patterns of search so that these designers can make search better.

The study of software design patterns owes its basic idea to Christopher Alexander, who wanted a language to help him describe the architecture of living spaces. In A Pattern Language, he recognized that how you put the patterns together makes all the difference. He analogized this to poetry. In prose we strive for simple sentences in which each word has an unambiguous meaning. In poetry the words come from the same language, but they have multiple meanings and connotations. They are packed densely together to express what prose cannot express efficiently.

Morville and Callender see poetry as the model for using the search patterns they present as the heart of their book. But they take the poetry model further. This is a densely packed book of interconnected meanings. Its 192 pages contain more useful information than many books of three times its length, but you must read slowly and carefully to extract that information. For example, here is how the authors identify the problem that they are trying to solve:

" . . . search is a wicked problem with no definitive formulation, considerable uncertainty, and complex interdependencies. Stakeholders have divergent goals and radically different world views. Requirements are incomplete, contradictory, and ever changing . . .

"Unlike Google, most firms aren't structured to manage the high-tech, step-changing, cross-functional, user-centered challenge. There are too many hyphens. As a hybrid of engineering, marketing, and design, search creates too many openings for missing links . . . Why risk your career (and your weekends) on a problem that's so intractable?

"We underfund and understaff search, and its poverty becomes a self-fulfilling prophecy. Users don't search now, because search failed then. Sometimes they browse. Often they bail."

To help describe patterns, the authors first break down search into its static anatomy and its behavioral aspects. The major elements of search are users, creators, content, the engine, and the interface. The many relationships among these elements help explain the complexity of search and the ideas in this book. For example, a search engine may take account of users' preferences, but if users don't bother to use the available customizations, they all wind up with the same default behavior. An interface may present results effectively, but if the searched content is full of redundant, outdated, trivial (ROT) material, users will not be able to find useful results quickly. Sometimes users are creators, when they give feedback about the relevance of results or comment upon a discovered item. The well known site that puts all of the elements together most effectively is amazon.com. The authors use Amazon as a good example of many patterns, but they point out that every situation is different. The Amazon approach works well because Amazon tailored its approach to the specific characteristics of its unique business.

The static anatomy helps us understand the elements of search, but to make specific situations better, designers must also understand the kind of conversation that leads to discovery in their environments. To succeed, search must have aspects of jazz. The process must flow. The authors cite Mihaly Csikszentmihalyi's contention that music, dancing, sailing, and chess are conducive to flow because they offer challenge, give control, support learning, reward skill, and provide feedback.

Just as designers repeat successful design patterns, users repeat the behavioral patterns that have worked for them. Search designers must support these behavior patterns. The main ways users interact iteratively with search facilities are narrowing/expanding searches and growing pearls. The main problems (antipatterns) they encounter are pogosticking and thrashing. To grow pearls, users find a good document and mine it for query terms and leads. Google supports this by offering links to similar items. Pogosticking is bouncing back and forth between a results page and the pages it points to. Thrashing occurs when users start from a flawed query (for example, using a misspelled term), then keep trying variations of it rather than switching to a better query.

Principles and patterns

The authors are interaction designers, and because they see little evidence of interaction design principles at work in search interfaces, they feel compelled to explain those principles. You'll have to read the book for the complete story, but here is a brief summary of some interaction design principles:

• Incremental construction lets users start small and build.
• Progressive disclosure hides complexity until it is needed.
• Direct manipulation provides visible objects for users to interact with.

The heart of the book is a discussion of ten design patterns. The choice of these specific patterns is meant as a convenient and somewhat arbitrary starting point. The authors describe each pattern, identify its most suitable contexts, and show how it relates to the other patterns. They discuss why the problem that the pattern addresses is common and why the pattern provides a good solution. They provide examples that you can visit, interact with, and study.

The Autocomplete pattern is the first and the easiest to understand. Typing is slow, especially on mobile devices. Users cannot spell, and they cannot remember search terms. As users enter the first few characters, the system begins to offer suggested search terms. This depends, of course, on having a good source of suggestions. Google does this well. Most sites do not.

The Best First pattern is deceptively simple. The system sorts results in such away that the first few results that users see are the ones most likely to be what they are looking for. A huge portion of what Google does behind the scenes supports this pattern. When implemented properly, a best first approach quickly gives users what they need or helps them formulate effective follow-up queries.

The Federated Search pattern addresses the problem of finding content that is distributed among a variety of content sources (silos). Implementing it entails problems such as slow performance and different query languages in different silos. Combining the silos into a single source may be a better approach to the problem. A good example of where to use federated search is hunting for books by author, title, subject, and so forth, across a set of independent university libraries.

The Pagination pattern emerged by accident, not as a response to a specific problem. Designers found it convenient to return a linear list of search results and to enable users to navigate through the list in screen-sized pages. Users soon discovered that pages provide a way to bookmark specific sets of results. Interfaces that replace the simple results page may have to provide a different approach to that use case.

The Faceted Navigation pattern gets away from the simple results page. It arises from the work of Marti Hearst and her Flamenco project at UC Berkeley. Using content and metadata from returned results, it constructs a custom visual map that makes it easy for users to narrow their queries incrementally. A drawback of this pattern is that it requires a large amount of screen space, so it does not work well on small mobile devices. The authors regard this pattern as the most significant search innovation of the past decade.

The Structured Results, Actionable Results, and Unified Discovery patterns all contribute to speeding discovery and making the process more iterative and interactive.

Moving search forward

Practitioners will especially appreciate the chapter called Engines of Discovery. Having laid out the principles of interaction design and created a language for discussing search patterns, the authors examine and discuss a number of sites that explore new ways to help their users discover what they may not have known they were looking for. Visiting these sites and understanding how they use search patterns is key to deriving maximum benefit from this book.

The authors hope that this book will influence the future of search technology. They see themselves not just as technologists and communicators, but as persuaders. In support of that function they have created some futuristic scenarios that they hope will serve as mythic guides for search technologists. These incorporate neural implants, direct communication of raw emotion, artificial intelligence, rogue agents, and software to recognize and track people.

This book is a monumental achievement. The authors have provided a guidebook to a largely unexplored and immensely important territory. Reading it requires attention, effort, and exploration. Not reading it is likely to lead you out to pasture.

Technical Writing

This article appears in slightly different form in the January/February 2010 issue of IEEE Micro © 2010 IEEE.

Before and during the dot-com boom, technical writing jobs in the US were plentiful and well paid. The ability to write coherent English sentences and use packaged software was sufficient qualification. Training in technical communication was not required, but certificate programs blossomed, and many new college graduates entered those programs. With the dot-com bust and the availability of high-speed communication between the US and India, many technical writing jobs became commodities and rapidly moved offshore. The number of people in the US identifying themselves as technical writers suddenly exceeded the available work. That number is now shrinking sharply to fit the demand. Many certificate programs have closed for lack of students.

Long before the dot-com boom and bust, however, a small cadre of academics and reflective practitioners established the underpinnings of a profession of technical communication. They showed how to determine the needs of their audiences and studied the most effective ways to satisy those needs. Some learned how to quantify the value that their work added to the products they documented. A few even learned the language of the business executives who determine the value that their organizations place on documentation. These academics and practitioners mostly survived, and they form the core of the technical communication workforce that exists today. Many, including me, are members of the Society for Technical Communication (STC), the leading professional organization for technical writers, as well as the IEEE Professional Communication Society and ACM's SIGDOC. A variety of online communities and mailing lists also support the field.

To be a successful technical writer today, you must understand the technical and business aspects of the work you do and how it affects the organizations who pay for that work. The book I look at this time covers all of the bases you need to touch.

Managing Writers: A Real World Guide to Managing Technical Documentation by Richard Hamilton (XML Press, Fort collins CO, 2009, 276pp, ISBN 978-0-9822191-0-2, xmlpress.net, 24.95)

Richard Hamilton began his career as a software developer in an organization that took software development seriously. His organization moved him directly from leading software teams to managing documentation, because they had begun to take documentation seriously as well. In the nearly 20 years since his first documentation management job, Hamilton has learned a lot about technical communication and how it fits into organizations like AT&T, Novell, and Hewlett-Packard. He has become an expert at applying XML technology to documentation projects and serves as a member of the OASIS DocBook Technical Committee. He is a member of STC.

Hamilton approaches the subject from the viewpoint of a manager, and he carefully covers everything that a technical writing manager should know. But writers who are not managers need this information as well. Technical writers in today's environment must understand the product, project, technology, and business issues. They must be comfortable communicating with the top leaders of their organizations, or those leaders are likely to regard them as commodities and their work as a cost that comes directly from the bottom line. On the first page of his book, describing the challenges facing technical writing managers, Hamilton says

Most important, technical documentation is viewed with disdain by many engineers and lives at the bottom of the power hierarchy in most companies. A significant amount of your time as a documentation manager will be spent working to gain respect, power, and leverage so you can do your job.

This sad fact applies to individual contributors as well. They must earn respect for themselves as members of the technical staff and of their project teams.

Hamilton gives an example of a situation in which he built power and influence. His firm issued annual updates of a complex product. The development groups were responsible for writing parts of a document describing their ongoing changes for the benefit of the support organization. When the writers needed clarification of the change document so that they could write clear release notes at the end of the year, the developers didn't want to provide information they thought they had already provided about features they had finished with months earlier. Hamilton's team took over preparing the ongoing change document, combined it with the release notes, and created an intranet site to simplify the developers' task of providing the new information and reviewing what the writers did with it. Hamilton's team thus gained control of the inputs they needed and won respect and gratitude for simplifying the developers' jobs and providing the support organization with clearer information.

Hiring and managing writers

Hamilton begins his book with a short summary of what technical writers do and the challenges they face. In a few pages he lays out his list of the key elements of technical writing: product, developers, audience, tasks, deliverables, environment, and schedules. Many writers focus on a few of these elements but ignore or skimp on others. Hamilton explains why each element is important and touches on the key issues associated with each of them.

Hamilton focuses on the three main aspects of managing: people, projects, and technology. These discussions become successively more abstract, though none of them strays far from concrete real-world situations. The discussion of managing people, however, seems close to Hamilton's heart. The examples are gripping; they deal with situations that should resonate at an emotional level with most readers.

Hamilton's longest chapter by far is on hiring, which shows the importance he attaches to that key activity. I have reviewed a number of books that talk about résumés. They present different, often contradictory, points of view. The message seems to be that there is more than one way to skin a cat. Hamilton details his process for evaluating résumés, and you won't go far wrong if you use it. He focuses on general themes rather than little details, and he shows a laudable level of flexibility. For example, he says he would hire a writer with only Windows experience to work on a Linux project, but he'd be reluctant to hire a writer who specializes in software to write an airplane hardware maintenance manual.

Hamilton makes a good point: the qualities that make a great technical writer boil down to the ability to understand and the ability to communicate. This seems obvious, but if you look at job ads, they often focus on tools, technology, and prior experience.

Hamilton is aware of the legal complications of hiring, managing, and occasionally firing employees. He contrasts the situation nostalgically with his father's situation as a personnel manager from the 1940s to the 1970s. He suggests making friends with people in the human relations department long before issues arise. This is the same advice Hamilton gives about building relationships with developers and with other managers. The pattern seems to be that good relationships lubricate formal processes. This is a tough lesson for introverts, a category that many technical writers inhabit. Many introverts would rather deal with processes than people.

Managing change

One of the most difficult problems of managing people concerns change. Most people adhere to the maxim, "If it ain't broke, don't fix it." It usually takes real pain to make people want to change. For example, if your company must translate your documentation into other languages, the potential costs are painful to the company's finances. Hamilton uses former General Electric chairman Jack Welch's metaphor of a burning deep sea platform. The managers standing on the burning translation costs platform feel the pain and are ready to act, but the writers may see it as someone else's problem as they watch from shore. The people who must make the change work must first understand and feel the need to do so.

Another common problem with managing change is to confuse the means with the ends. Hamilton tells us to define the desired future state in terms that impose minimal constraints on the means you use to get there. In particular, that means keeping solution vendors -- and consultants who are tied to specific vendors -- out of the process until you have a clear idea of where you are going.

Managing technology

New technologies present technical as well as personnel problems. Hamilton provides good advice about managing technology. For example, not every problem can be solved by technology. If you have badly structured content, don't count on a content management system to fix it. Figure out what you are trying to do and what obstacles are causing your pain. Articulate the desired final state. Then look for a solution, which may or may not involve new technology.

Once you have a solution, Hamilton suggests easing the personnel problems associated with the pain of adopting the change by phasing it in, providing good training to everybody, and keeping management in the loop -- especially when problems arise. It is better for them to hear about problems from you before they hear from your critics. One aspect of phasing in change is to start with people who have a positive attitude toward the change. Get the bugs out before bringing the troglodytes along.

Hamilton is a specialist in using XML technology, but he is also aware of the hype associated with it. He quotes Richard Feynman: "For a successful technology, reality must take precedence over public relations." The reality in this case is that it "will not organize your documentation, eliminate your production back-end, nor allow you to hire fewer, less skilled, or cheaper writers." XML is a system for producing markup languages (for example, DocBook or DITA). Such languages, when they integrate well with the structure of your documentation, help make your content easy to search, manipulate, reuse, and repurpose. Hamilton provides useful guidance in addressing these issues. There is no magic. You may have to make substantial investments before you begin to realize any of the benefits.

Many technical writing managers must deal with complex problems like single sourcing and localization. For such problems, the solution often includes some sort of content management system. Hamilton explores the issues involved in setting up processes to support single sourcing and localization. He also talks about how to decide what sort of technological help you need in managing content.

While many vendors cite reuse as one of the main purposes of a content management system, Hamilton advises you to start by organizing your content in such a way as to minimize reuse. He considers it better to have just one logical place to look for a piece of content than it is to have it in different places, even if you have a system for synchronizing the content at those different places.

Resisting the dark side

One reason I am so enthusiastic about this book is that Hamilton is not in awe of sacred cows. Two of them are performance evaluation and performance metrics.

Business success depends on clear expectations between managers and their employees. They need a framework for ongoing discussion of goals, objectives, and other workplace issues. Weekly one-on-one meetings often provide this framework. On top of that, however, most companies use an annual performance evaluation process. This could be a beneficial way to focus on long term issues, but most companies use it for a different purpose: sorting employees into winners and losers. This robs the process of many good effects it might theoretically have had. Neither managers nor employees can engage openly and honestly in the process, because they are aware of its negative consequences.

Nonetheless, Hamilton recognizes that almost all companies have a performance evaluation process. Eisenhower said something to the effect that plans are useless but planning is indispensable. Hamilton applies that model to performance evaluation. He does what he can to make the process of performance evaluation helpful. He suggests ways to do as little harm as possible with the resulting documents and the ensuing discussions with employees.

Hamilton devotes approximately a quarter of his book to managing documentation projects. He makes points I haven't seen in substantially longer works on the subject, but I especially like his discussion of metrics. There is a cliché to the effect that if you don't measure something you can't manage it. Furthermore, keeping good records of predicted and actual values of some metrics helps to improve planning and estimating. Despite their benefits, however, metrics have downsides. Writers are aware that management may use these metrics to evaluate performance, so the measured aspects improve to the detriment of the unmeasured aspects. Unfortunately, the easiest aspects to measure are not likely to be the most important. For example, it is easy to count pages and hard to measure quality.

As with performance evaluations, Hamilton shows how to make the best of a bad tool. He delegates measurement to individual writers to use as they see fit, and he solemnly promises not to use those measurements to reward or punish writers -- or report the measurements to anyone likely to use them in that way.

Hamilton has much more to say about managing people, projects and technology. This short book is full of useful information and hard-earned opinions. Whether you manage writers or would rather just sit in your cubicle and write, you need to read this book. It is the most practical and realistic book I have seen on the subject, and I recommend it highly.