Sunday, March 21, 2010

Search Patterns

This article appears in slightly different form in the March/April 2010
issue of IEEE Micro © 2010 IEEE.


Designing for Discovery

Asking questions and wanting to know the answers is basic to human nature. No one knows of a time when this was not so. Researchers have speculated that human culture took a great leap forward when life expectancies increased to the point that children could interact with their grandparents. Word of mouth sufficed for a long time, but writing and printing led to further leaps forward.

It is a commonplace observation that sources of data are now increasing exponentially and that old methods of transmitting knowledge cannot keep up. Search technology has arisen to address this problem, but how best to use that technology remains an unsolved problem. Most computer users know how to go to Google or other search engines, enter search terms, and browse through results. Few user interface designers, however, know how to incorporate effective search technology into their designs. The book I look at this time addresses that problem.


Search Patterns: Design for Discovery by Peter Morville and Jeffery Callender (O'Reilly, Sebastopol CA, 2010, 192pp, ISBN 978-0-596-80227-1 www.oreilly.com, $39.99)

Peter Morville is a founder of the field of information architecture. He is author of "Information Architecture for the World Wide Web" and "Ambient Findability." His consulting clients include AT&T, Harvard, IBM, Microsoft, and the Library of Congress. Jeffrey Callender is the design director of Q LTD, a strategic design consultantly with clients worldwide. Peter bills himself as the word person. Jeff is the visual thinker. Their collaboration successfully integrates these ways of thinking.

Search, say the authors, is the worst usability problem on the web. We go to a website to find something -- a fact, a product, background information, an old acquaintance. Often, what we thought we came for is not actually what we need. Search is a conversation in which we discover new things and ask new questions. How quickly and accurately we find what we need defines the quality of our experience. If we give up and revert to more costly options like telephone or email, everyone loses. Yet that is what often happens when companies fail to use search technology effectively.

Patterns and interconnections

Apophenia, an aspect of madness and creativity, is the ability to see connections that others don't see. The savant Daniel Tammet sees patterns of shape and color in the digits of π that allow him to recite tens of thousands of digits accurately. To those who do not perceive such patterns, the feat seems impossible.

The authors are writing for an audience of highly sophisticated user interface designers who can hold their own with engineers and marketers and make the business case with upper management for investment in search technology. The authors' goal is to discover and study the patterns of search so that these designers can make search better.

The study of software design patterns owes its basic idea to Christopher Alexander, who wanted a language to help him describe the architecture of living spaces. In A Pattern Language, he recognized that how you put the patterns together makes all the difference. He analogized this to poetry. In prose we strive for simple sentences in which each word has an unambiguous meaning. In poetry the words come from the same language, but they have multiple meanings and connotations. They are packed densely together to express what prose cannot express efficiently.

Morville and Callender see poetry as the model for using the search patterns they present as the heart of their book. But they take the poetry model further. This is a densely packed book of interconnected meanings. Its 192 pages contain more useful information than many books of three times its length, but you must read slowly and carefully to extract that information. For example, here is how the authors identify the problem that they are trying to solve:
" . . . search is a wicked problem with no definitive formulation, considerable uncertainty, and complex interdependencies. Stakeholders have divergent goals and radically different world views. Requirements are incomplete, contradictory, and ever changing . . .

"Unlike Google, most firms aren't structured to manage the high-tech, step-changing, cross-functional, user-centered challenge. There are too many hyphens. As a hybrid of engineering, marketing, and design, search creates too many openings for missing links . . . Why risk your career (and your weekends) on a problem that's so intractable?

"We underfund and understaff search, and its poverty becomes a self-fulfilling prophecy. Users don't search now, because search failed then. Sometimes they browse. Often they bail."
To help describe patterns, the authors first break down search into its static anatomy and its behavioral aspects. The major elements of search are users, creators, content, the engine, and the interface. The many relationships among these elements help explain the complexity of search and the ideas in this book. For example, a search engine may take account of users' preferences, but if users don't bother to use the available customizations, they all wind up with the same default behavior. An interface may present results effectively, but if the searched content is full of redundant, outdated, trivial (ROT) material, users will not be able to find useful results quickly. Sometimes users are creators, when they give feedback about the relevance of results or comment upon a discovered item. The well known site that puts all of the elements together most effectively is amazon.com. The authors use Amazon as a good example of many patterns, but they point out that every situation is different. The Amazon approach works well because Amazon tailored its approach to the specific characteristics of its unique business.

The static anatomy helps us understand the elements of search, but to make specific situations better, designers must also understand the kind of conversation that leads to discovery in their environments. To succeed, search must have aspects of jazz. The process must flow. The authors cite Mihaly Csikszentmihalyi's contention that music, dancing, sailing, and chess are conducive to flow because they offer challenge, give control, support learning, reward skill, and provide feedback.

Just as designers repeat successful design patterns, users repeat the behavioral patterns that have worked for them. Search designers must support these behavior patterns. The main ways users interact iteratively with search facilities are narrowing/expanding searches and growing pearls. The main problems (antipatterns) they encounter are pogosticking and thrashing. To grow pearls, users find a good document and mine it for query terms and leads. Google supports this by offering links to similar items. Pogosticking is bouncing back and forth between a results page and the pages it points to. Thrashing occurs when users start from a flawed query (for example, using a misspelled term), then keep trying variations of it rather than switching to a better query.

Principles and patterns

The authors are interaction designers, and because they see little evidence of interaction design principles at work in search interfaces, they feel compelled to explain those principles. You'll have to read the book for the complete story, but here is a brief summary of some interaction design principles:
  • Incremental construction lets users start small and build.
  • Progressive disclosure hides complexity until it is needed.
  • Direct manipulation provides visible objects for users to interact with.
The heart of the book is a discussion of ten design patterns. The choice of these specific patterns is meant as a convenient and somewhat arbitrary starting point. The authors describe each pattern, identify its most suitable contexts, and show how it relates to the other patterns. They discuss why the problem that the pattern addresses is common and why the pattern provides a good solution. They provide examples that you can visit, interact with, and study.

The Autocomplete pattern is the first and the easiest to understand. Typing is slow, especially on mobile devices. Users cannot spell, and they cannot remember search terms. As users enter the first few characters, the system begins to offer suggested search terms. This depends, of course, on having a good source of suggestions. Google does this well. Most sites do not.

The Best First pattern is deceptively simple. The system sorts results in such away that the first few results that users see are the ones most likely to be what they are looking for. A huge portion of what Google does behind the scenes supports this pattern. When implemented properly, a best first approach quickly gives users what they need or helps them formulate effective follow-up queries.

The Federated Search pattern addresses the problem of finding content that is distributed among a variety of content sources (silos). Implementing it entails problems such as slow performance and different query languages in different silos. Combining the silos into a single source may be a better approach to the problem. A good example of where to use federated search is hunting for books by author, title, subject, and so forth, across a set of independent university libraries.

The Pagination pattern emerged by accident, not as a response to a specific problem. Designers found it convenient to return a linear list of search results and to enable users to navigate through the list in screen-sized pages. Users soon discovered that pages provide a way to bookmark specific sets of results. Interfaces that replace the simple results page may have to provide a different approach to that use case.

The Faceted Navigation pattern gets away from the simple results page. It arises from the work of Marti Hearst and her Flamenco project at UC Berkeley. Using content and metadata from returned results, it constructs a custom visual map that makes it easy for users to narrow their queries incrementally. A drawback of this pattern is that it requires a large amount of screen space, so it does not work well on small mobile devices. The authors regard this pattern as the most significant search innovation of the past decade.

The Structured Results, Actionable Results, and Unified Discovery patterns all contribute to speeding discovery and making the process more iterative and interactive.

Moving search forward

Practitioners will especially appreciate the chapter called Engines of Discovery. Having laid out the principles of interaction design and created a language for discussing search patterns, the authors examine and discuss a number of sites that explore new ways to help their users discover what they may not have known they were looking for. Visiting these sites and understanding how they use search patterns is key to deriving maximum benefit from this book.

The authors hope that this book will influence the future of search technology. They see themselves not just as technologists and communicators, but as persuaders. In support of that function they have created some futuristic scenarios that they hope will serve as mythic guides for search technologists. These incorporate neural implants, direct communication of raw emotion, artificial intelligence, rogue agents, and software to recognize and track people.

This book is a monumental achievement. The authors have provided a guidebook to a largely unexplored and immensely important territory. Reading it requires attention, effort, and exploration. Not reading it is likely to lead you out to pasture.



Sunday, January 24, 2010

Technical Writing

This article appears in slightly different form in the January/February 2010 issue of IEEE Micro © 2010 IEEE.


Before and during the dot-com boom, technical writing jobs in the US were plentiful and well paid. The ability to write coherent English sentences and use packaged software was sufficient qualification. Training in technical communication was not required, but certificate programs blossomed, and many new college graduates entered those programs. With the dot-com bust and the availability of high-speed communication between the US and India, many technical writing jobs became commodities and rapidly moved offshore. The number of people in the US identifying themselves as technical writers suddenly exceeded the available work. That number is now shrinking sharply to fit the demand. Many certificate programs have closed for lack of students.

Long before the dot-com boom and bust, however, a small cadre of academics and reflective practitioners established the underpinnings of a profession of technical communication. They showed how to determine the needs of their audiences and studied the most effective ways to satisy those needs. Some learned how to quantify the value that their work added to the products they documented. A few even learned the language of the business executives who determine the value that their organizations place on documentation. These academics and practitioners mostly survived, and they form the core of the technical communication workforce that exists today. Many, including me, are members of the Society for Technical Communication (STC), the leading professional organization for technical writers, as well as the IEEE Professional Communication Society and ACM's SIGDOC. A variety of online communities and mailing lists also support the field.

To be a successful technical writer today, you must understand the technical and business aspects of the work you do and how it affects the organizations who pay for that work. The book I look at this time covers all of the bases you need to touch.

Managing Writers: A Real World Guide to Managing Technical Documentation by Richard Hamilton (XML Press, Fort collins CO, 2009, 276pp, ISBN 978-0-9822191-0-2, xmlpress.net, 24.95)

Richard Hamilton began his career as a software developer in an organization that took software development seriously. His organization moved him directly from leading software teams to managing documentation, because they had begun to take documentation seriously as well. In the nearly 20 years since his first documentation management job, Hamilton has learned a lot about technical communication and how it fits into organizations like AT&T, Novell, and Hewlett-Packard. He has become an expert at applying XML technology to documentation projects and serves as a member of the OASIS DocBook Technical Committee. He is a member of STC.

Hamilton approaches the subject from the viewpoint of a manager, and he carefully covers everything that a technical writing manager should know. But writers who are not managers need this information as well. Technical writers in today's environment must understand the product, project, technology, and business issues. They must be comfortable communicating with the top leaders of their organizations, or those leaders are likely to regard them as commodities and their work as a cost that comes directly from the bottom line. On the first page of his book, describing the challenges facing technical writing managers, Hamilton says
Most important, technical documentation is viewed with disdain by many engineers and lives at the bottom of the power hierarchy in most companies. A significant amount of your time as a documentation manager will be spent working to gain respect, power, and leverage so you can do your job.
This sad fact applies to individual contributors as well. They must earn respect for themselves as members of the technical staff and of their project teams.

Hamilton gives an example of a situation in which he built power and influence. His firm issued annual updates of a complex product. The development groups were responsible for writing parts of a document describing their ongoing changes for the benefit of the support organization. When the writers needed clarification of the change document so that they could write clear release notes at the end of the year, the developers didn't want to provide information they thought they had already provided about features they had finished with months earlier. Hamilton's team took over preparing the ongoing change document, combined it with the release notes, and created an intranet site to simplify the developers' task of providing the new information and reviewing what the writers did with it. Hamilton's team thus gained control of the inputs they needed and won respect and gratitude for simplifying the developers' jobs and providing the support organization with clearer information.

Hiring and managing writers

Hamilton begins his book with a short summary of what technical writers do and the challenges they face. In a few pages he lays out his list of the key elements of technical writing: product, developers, audience, tasks, deliverables, environment, and schedules. Many writers focus on a few of these elements but ignore or skimp on others. Hamilton explains why each element is important and touches on the key issues associated with each of them.

Hamilton focuses on the three main aspects of managing: people, projects, and technology. These discussions become successively more abstract, though none of them strays far from concrete real-world situations. The discussion of managing people, however, seems close to Hamilton's heart. The examples are gripping; they deal with situations that should resonate at an emotional level with most readers.

Hamilton's longest chapter by far is on hiring, which shows the importance he attaches to that key activity. I have reviewed a number of books that talk about résumés. They present different, often contradictory, points of view. The message seems to be that there is more than one way to skin a cat. Hamilton details his process for evaluating résumés, and you won't go far wrong if you use it. He focuses on general themes rather than little details, and he shows a laudable level of flexibility. For example, he says he would hire a writer with only Windows experience to work on a Linux project, but he'd be reluctant to hire a writer who specializes in software to write an airplane hardware maintenance manual.

Hamilton makes a good point: the qualities that make a great technical writer boil down to the ability to understand and the ability to communicate. This seems obvious, but if you look at job ads, they often focus on tools, technology, and prior experience.

Hamilton is aware of the legal complications of hiring, managing, and occasionally firing employees. He contrasts the situation nostalgically with his father's situation as a personnel manager from the 1940s to the 1970s. He suggests making friends with people in the human relations department long before issues arise. This is the same advice Hamilton gives about building relationships with developers and with other managers. The pattern seems to be that good relationships lubricate formal processes. This is a tough lesson for introverts, a category that many technical writers inhabit. Many introverts would rather deal with processes than people.

Managing change

One of the most difficult problems of managing people concerns change. Most people adhere to the maxim, "If it ain't broke, don't fix it." It usually takes real pain to make people want to change. For example, if your company must translate your documentation into other languages, the potential costs are painful to the company's finances. Hamilton uses former General Electric chairman Jack Welch's metaphor of a burning deep sea platform. The managers standing on the burning translation costs platform feel the pain and are ready to act, but the writers may see it as someone else's problem as they watch from shore. The people who must make the change work must first understand and feel the need to do so.

Another common problem with managing change is to confuse the means with the ends. Hamilton tells us to define the desired future state in terms that impose minimal constraints on the means you use to get there. In particular, that means keeping solution vendors -- and consultants who are tied to specific vendors -- out of the process until you have a clear idea of where you are going.

Managing technology

New technologies present technical as well as personnel problems. Hamilton provides good advice about managing technology. For example, not every problem can be solved by technology. If you have badly structured content, don't count on a content management system to fix it. Figure out what you are trying to do and what obstacles are causing your pain. Articulate the desired final state. Then look for a solution, which may or may not involve new technology.

Once you have a solution, Hamilton suggests easing the personnel problems associated with the pain of adopting the change by phasing it in, providing good training to everybody, and keeping management in the loop -- especially when problems arise. It is better for them to hear about problems from you before they hear from your critics. One aspect of phasing in change is to start with people who have a positive attitude toward the change. Get the bugs out before bringing the troglodytes along.

Hamilton is a specialist in using XML technology, but he is also aware of the hype associated with it. He quotes Richard Feynman: "For a successful technology, reality must take precedence over public relations." The reality in this case is that it "will not organize your documentation, eliminate your production back-end, nor allow you to hire fewer, less skilled, or cheaper writers." XML is a system for producing markup languages (for example, DocBook or DITA). Such languages, when they integrate well with the structure of your documentation, help make your content easy to search, manipulate, reuse, and repurpose. Hamilton provides useful guidance in addressing these issues. There is no magic. You may have to make substantial investments before you begin to realize any of the benefits.

Many technical writing managers must deal with complex problems like single sourcing and localization. For such problems, the solution often includes some sort of content management system. Hamilton explores the issues involved in setting up processes to support single sourcing and localization. He also talks about how to decide what sort of technological help you need in managing content.

While many vendors cite reuse as one of the main purposes of a content management system, Hamilton advises you to start by organizing your content in such a way as to minimize reuse. He considers it better to have just one logical place to look for a piece of content than it is to have it in different places, even if you have a system for synchronizing the content at those different places.

Resisting the dark side

One reason I am so enthusiastic about this book is that Hamilton is not in awe of sacred cows. Two of them are performance evaluation and performance metrics.

Business success depends on clear expectations between managers and their employees. They need a framework for ongoing discussion of goals, objectives, and other workplace issues. Weekly one-on-one meetings often provide this framework. On top of that, however, most companies use an annual performance evaluation process. This could be a beneficial way to focus on long term issues, but most companies use it for a different purpose: sorting employees into winners and losers. This robs the process of many good effects it might theoretically have had. Neither managers nor employees can engage openly and honestly in the process, because they are aware of its negative consequences.

Nonetheless, Hamilton recognizes that almost all companies have a performance evaluation process. Eisenhower said something to the effect that plans are useless but planning is indispensable. Hamilton applies that model to performance evaluation. He does what he can to make the process of performance evaluation helpful. He suggests ways to do as little harm as possible with the resulting documents and the ensuing discussions with employees.

Hamilton devotes approximately a quarter of his book to managing documentation projects. He makes points I haven't seen in substantially longer works on the subject, but I especially like his discussion of metrics. There is a cliché to the effect that if you don't measure something you can't manage it. Furthermore, keeping good records of predicted and actual values of some metrics helps to improve planning and estimating. Despite their benefits, however, metrics have downsides. Writers are aware that management may use these metrics to evaluate performance, so the measured aspects improve to the detriment of the unmeasured aspects. Unfortunately, the easiest aspects to measure are not likely to be the most important. For example, it is easy to count pages and hard to measure quality.

As with performance evaluations, Hamilton shows how to make the best of a bad tool. He delegates measurement to individual writers to use as they see fit, and he solemnly promises not to use those measurements to reward or punish writers -- or report the measurements to anyone likely to use them in that way.

Hamilton has much more to say about managing people, projects and technology. This short book is full of useful information and hard-earned opinions. Whether you manage writers or would rather just sit in your cubicle and write, you need to read this book. It is the most practical and realistic book I have seen on the subject, and I recommend it highly.

Wednesday, October 7, 2009

Life and Work

This article appears in slightly different form in the September/October 2009 issue of
IEEE Micro © 2009 IEEE.



The Declaration of Independence of the United States of America declares that every person has the right to life, liberty, and the pursuit of happiness. These are precious rights, but exercising them takes work. Pursuing happiness entails overcoming many obstacles. The easiest and hardest of these are often in our own heads.

Recently I attended a workshop called Strategic Planning for Your Life, led by Judy Glick-Smith, a former president and Fellow of the Society for Technical Communication (STC). That workshop is not the subject of this review. You can find information about it at www.mentorfactorinc.com. Glick-Smith's method asks you to begin by discovering what makes you happy and casting aside the internal obstacles to pursuing that happiness. She then lays out steps whereby you can arrive at a set of life goals and an ongoing process for pursuing and updating them.

By whatever path you arrive at your life goals, they are likely to include some sort of productive work. If that work is a high-tech job, I can recommend a book to help you.


Land the Tech Job You Love by Andy Lester (Pragmatic Bookshelf, Raleigh NC, 2009, 272pp, ISBN 978-1-93435-626-5, www.pragprog.com, $23.95)

Andy Lester has been writing software since the mid-1980s. He is active in the Perl community, where he maintains more than 20 modules on the Comprehensive Perl Archive Network (CPAN). In the course of his professional life he has seen countless résumés, interviewed many candidates, and done his share of job hunting as well. His frustration at what he has seen led him to write this book.

Many people give job-hunting advice. Most are dogmatic about their recommendations, even though one effective job hunter may give you rules that contradict those of another equally effective job hunter. Lester is no exception. He is sure of his rules, though he recognizes that they may not apply to your situation. Hardly anybody, however, is likely to disagree with the basic principles he starts with:
  • Be honest with yourself.
  • Be honest with others.
  • Think like the boss.
  • Be a problem solver.
  • Sell yourself.
  • Tell stories.
  • Be positive.
Lester begins from the premise that life is too short for a job you don't love. Being honest with yourself is a key to ensuring that you don't wind up in a job that you don't love. Lester urges you to think about what you want in a job before you start looking at job ads or writing a résumé. Following one of his own principles, he tells a story of how he had taken a seemingly perfect job and found it to be unbearable. He hadn't aksed himself what he really wanted in the job. Had he done so and given himself honest answers, he would not have taken the job.

Once you decide what you want in a job, it's time to think about a résumé. Lester identifies what he considers to be the basic sections of a résumé and tells you how to gather and organize the contents of each. There are no surprises here, but his systematic coverage can help you ensure that you haven't overlooked anything.

Lester also identifies material that you should not include. Some of this is uncontroversial -- for example, don't include a photo or any information that you can't legally be asked about in an interview. Cautious hiring managers will discard such résumés without looking further. Some of his other rules, however, may cause raised eyebrows. Some experts suggest beginning a résumé with an objective, but Lester considers objectives to be useless fluff, often filled with meaningless phrases like "challenging position" and "contribute my skills." And if you ever apply for a job for which Andy Lester is the hiring manager, don't waste space on your résumé with the phrase "references available on request."

Once you have the basic building blocks, Lester wants you to put them together in a variety of ways. Of course you need a generic résumé on paper and in electronic form, but that's just the beginning. Ideally, for example, you should create a separate résumé for each position, tailoring and arranging the parts to highlight the ways in which you meet that job's requirements. Furthermore, you need versions in various formats. Some hiring managers ask for résumés in Word format, while others want text, and you should also have a generic HTML version on a publicly accessible website. If someone asks for Word format and you send them PDF, they already know that you can't or won't follow directions. Lester has much more to say about résumés, and it's worth reading, even if you don't agree with all of it.

After all of this groundwork, you have to find a job to apply for. Lester wisely steers you away from "job boards," websites that treat candidates and jobs as fungible commodities to be matched with one another. He also distrusts recruiters, because their clients are the hiring firms. You might think that recruiters find positions for candidates, but their actual job is to find candidates for positions. Instead of using job boards and recruiters, ask the people you know -- including your Aunt Edna, but excluding your current co-workers -- to help. Help can take many forms, but Lester advises keeping it lightweight. Ask for pointers and leads that you can follow up on. Don't ask anyone to do your job hunting for you.

Another piece of good advice that Lester offers is to use traditional sources like newspapers or the local Chamber of Commerce. Newspapers can give you a high level view of what's available in the area they serve. The staff of the Chamber of commerce can probably give you pointers to valuable contacts inside firms that need your services.

Once you find some interesting job possibilities, there is more to do before you send anybody your résumé. Lester tells you how to use publicly available sources like a company's website or its Securities and Exchange Commission (SEC) filings to answer the key questions: how does this company make money, how could I help it make money, and would I enjoy doing so? Remember, life is too short for a job you don't love.

Lester suggests using online resources to help you find information about a company. You might discreetly approach a current or former employee who posts to a technical mailing list that you belong to. Or you might look on the Facebook or LinkedIn social networking sites for people whose profiles mention the company.

Once you've found out everything you can about the company, it's time to apply for the job. Many people will tell you to send your résumé with a cover letter, but Lester points out an aspect of cover letters that you may not have thought of. If you can't come up with two paragraphs that explain why you are a good match for the specific job and company, you probably shouldn't apply.

Jack Molisani, a recruiter who often makes presentations about job hunting, tells the story of a résumé he sent to a client who wanted someone with patent experience. The client called him back and asked why Molisani was wasting his time with a candidate who didn't have patent experience. The résumé mentioned the candidate's patent experience, but not near the top, and the client hadn't read far enough to find it. This illustrates several of Lester's points. One point is that if interviewers ask you questions that your résumé clearly answers, don't say "read the bleeping résumé." Molisani knew better than to call his client's attention to the material the client had overlooked.

A second point that Molisani's story illustrates is that you should look at your résumé through the eyes of the hiring manager. Think about what is important to the hiring manager and make that part of your experience prominent on the résumé. I agree with that point, but I disagree with the method Lester suggests for making parts of your résumé prominent. Lester suggests highlighting words of the résumé that are important to the specific job by making them bold. This is a dangerous technique. A skillful communicator might get away with it, but most job candidates are likely to wind up with résumés that look like a ransom notes. Other techniques that Lester also advocates are more effective. Rearrange the sections of the résumé to highlight the most important information, and mention key points in the cover letter. By all means, tailor the résumé and cover letter to the job, but go easy on the bold.

While Lester rightly encourages you to follow directions and give hiring managers what they ask for, he makes an exception for salary history. Never provide this information. It's nobody's business, and there is no benefit to providing it. Lester suggests finessing this request by politely but firmly noting that the information is confidential and that it's not a subject that you discuss with anyone.

When the company invites you to a job interview, you're almost there. Unfortunately, many candidates perform badly in interviews. The hiring manager is likely to prepare carefully for the interview, and you should do the same. This includes preparing logistically. Arrive on time with whatever materials you need to bring with you. Clear your schedule, so you can stay as long as necessary.

There are many ways to look at interviews. They are conversations in which you can lead as well as follow. They are sales pitches by which you show that you are the right person for the job and make the hiring manager eager for your services. They are a first day on the job in which you seek out the areas where the company needs you most. Unfortunately, they can also be an obstacle course in which the hiring manager seeks to trip you up with questions designed to eliminate candidates and simplify the hiring choice. Lester prepares you for all of these aspects of interviews.

One piece of advice that Lester provides applies to many of life's situations -- not just to interview questions. Listen, and seek to understand not just the question, but the reason for the question. Remember that an interview is a chance for each party to decide whether they would like to work with the other. Sometimes understanding the reasons for questions can help you with your half of that decision. And being sure you understand a question before you answer it can keep you from saying something that makes you look uninformed or ill-prepared.

Lester also provides a lot of advice that you've probably heard before, but it can't hurt to keep it in mind. Arrive a little early. Treat everbody like the CEO -- especially the receptionist. Stand to shake hands. Make eye contact (but don't stare). Be positive. Ask for the job.

I do not completely agree with Lester about how to handle a portfolio. I learned about how to use portfolios from Lance Gelein, another former president of STC. To Gelein a portfolio is a set of impressive props to use while telling stories. If the conversation comes around to a specific kind of job, you open your portfolio to a sample of that sort of work and tell the story of how you approached it, what obstacles you faced, and how you overcame them. The portfolio piece is not the entire job, but a snippet that you can point at while telling the story. Your portfolio does not stand alone without you. You never leave it with the hiring manager or let anyone look at it without your accompanying narration.

To Lester, a portfolio is a collection of representative pieces of work that you are proud of. Each stands alone, and you happily leave a copy behind for the hiring manager to peruse after you've left. There is, of course, a place for this sort of work sample, but it doesn't fully exploit the story-telling potential of a portfolio.

Lester devotes an entire chapter to tough interview questions. Again, the key is to understand the reason behind the question, so you can directly, without equivocation, satisfy the interviewer's concern. Lester identifies a number of red flags that interviewers want to uncover. Some of these are two-edged swords. For example, do you refuse to ask for help? Are you unable to work without constant hand holding? Interviewers ask questions designed to detect where you stand with respect to these extremes. If you recognize what they are getting at, you can give a useful answer. Don't try to duck questions like "What is your greatest weakness?" or "Tell me about a project that didn't go well." Lester shows you how to approach such questions.

Lester's advice doesn't stop with the interview. He covers follow-up, dealing with an offer or with rejection, and even how to stay hirable after you start the new job. He has written a short but comprehensive book -- easy to read and full of good advice. Like every other book about job hunting, this one is not perfect, but if you absorb its advice and adapt it to your situation, it should pay for itself long before your first hour on your new tech job is complete.

Tuesday, August 11, 2009

Twitter

This article appears in slightly different form in the July/August 2009 issue of
IEEE Micro © 2009 IEEE.




The Twitter social messaging service is simple, but hard to understand, at least at first. I poked around its edges for a while, until I found the Twitter Book.

The Twitter Book

The Twitter Book by Tim O'Reilly & Sarah Milstein (O'Reilly, Sebastopol CA, 2009, 240pp, ISBN 978-0-596-80281-3, www.oreily.com, $19.99)

Tim O'Reilly is the founder and CEO of O'Reilly Media. I have never before reviewed a book that he wrote, but I have been reviewing books that he published for about 20 years. Starting with a few how-to books for Unix users in the 1980s, O'Reilly Media has become the leading publisher of books about hot software topics. In recent years, Tim O'Reilly has become a leader in the open source and web 2.0 communities.

Sarah Milstein started out as a freelance writer and editor. She was a regular contributor to The New York Times. Later she joined O'Reilly media, where she led the development of the Missing Manual series. She now travels around the country, lecturing and teaching about Twitter. With fellow Times contributor Marci Alboher, Sarah is cofounder of 20slides.com, which will soon launch a series of career-related online workshops.

The Twitter Book has been my gateway to Twitter. It contains solid basic information about all aspects of Twitter and provides context for a large number of pointers. By following those pointers, I learned everything I know about Twitter. If you invest a few hours reading this clear and comprehensive book, you will be well on your way to understanding how Twitter fits into your life.

Basics of Twitter

To begin using Twitter, go to twitter.com and sign up. Supply your full name, and pick a username that is short and to the point. My username is xrmxrm, which includes my initials and echoes my email addresses and Facebook ID. No username that includes a significant indication of my full name would be short enough to be an effective Twitter username.

Next, set up your profile. Twitter allows you a picture, a 160-character biography and a location. My biography is:

Technical communicator specializing in documentation for programmers -- APIs, web services, programmer guides. Columnist for IEEE Micro.

My location is Berkeley, CA.

Your name, your location, your biography, and your contributions are all that a person considering following your contributions has to go on.

Before using Twitter, you should understand updates, following, @references, retweeting, hashcodes, and searching.

Updates, also called tweets, are your contributions. You can enter a message of up to 140 characters, and everyone who has signed up to follow you will see that message, with your picture, among the tweets that come to them. Anyone can follow your updates, unless you protect updates, which few people do, because there is little point to doing so. You can send updates from a browser or a third party Twitter client, on your computer or your cell phone.

Following is an easily established asymmetric relationship between you and another twitter user. You simply go to that user's page (twitter.com/their_username) and click the Follow button. You can follow other users without their permission and without their following you. Twitter will, if you choose, notify you by email when other users choose to follow you. If you like, you can view their pages and decide whether to follow them. Many people automatically follow people who are following them, but there is no need to do so. Some people follow you as a form of spam. Obviously, you should not follow them. Twitter provides an easy way for users to report spam.

The most interesting uses of Twitter are conversational, so you need a way to refer to other people's tweets. Twitter interprets content of the form @username as a link, called an @reference, to that user's page. By a quirk of Twitter, if you begin a tweet with an @reference, most Twitter users who follow you but do not follow the user referred to in the @reference will not see the tweet. As a result, if you want to pass a tweet to your followers, a process called retweeting, you should place the @reference in the body of the tweet. The standard formats for retweets are to begin with RT @username or to end with via @username.

Twitter has a powerful search mechanism. In addition, it automatically brings to your attention the topics that are most popular at the moment. Search results can contain the tweets of users you are not following, so searching enables you to follow interesting trends and find interesting users to follow.

Often, a group of users interested in a topic choose a search term to include in tweets about that topic. Such search terms usually begin with hash mark (#). These are called hashcodes. As I write this, #iranelection and #michaeljackson are widely used hashcodes. Nobody assigns hashcodes. Users simply start using them. Different versions soon converge, and conflicts work themselves out.

Twitter Helpers

Twitter provides a certain amount of pushed summary information. In addition, a number of third parties follow Twitter, massage the flow of tweets, and make the information easily available. They can notify you about matters of interest to you, so you don't need to read millions of tweets each day. For example, TwitScoop shows hot topics as a tag cloud, in which terms appear in larger or smaller font sizes, depending on their recent frequency.

Another difficult problem for most new Twitter users is deciding which users to follow. Sites like MrTweet provide personalized lists of Twitter users for you to consider following.

If you want others to find your tweets worth reading and retweeting, you should link to interesting material. URLs can be long, and Twitter's limit is 140 characters for your entire tweet, so you need to find a good URL shortener. Such services (for example, TinyURL, Bit.ly, is.gd, and Twi.bz) replace a long URL with a unique, permanent short one that points to the same place. Different shorteners provide different advantages. For example, some give you some control over the resulting URL, while others make it easy for you to track clicks.

Pictures are a compelling component of many tweets. You can post a picture to a site like flickr.com and use a shortened URL to point to it. Others can leave comments at the site or retweet the link with their comments. The twitpic.com site integrates especially well with Twitter.

If you interact with Twitter solely through twitter.com, the primitive user interface soon becomes limiting. You must refresh the page to see new tweets. Simple tasks like replying and retweeting entail tedious manual steps that cry out for automation. Because Twitter provides an application programming interface (API), many third parties have created programs to put a different front end on Twitter. Most such programs run on computers, but others work on smart phones. I have tried Twhirl, SeesmicDesktop, TweetDeck, and DestroyTwitter on my laptop. Each provides improvements over the twitter.com interface, but I keep coming back to twitter.com because I feel more comfortable and less restricted there.

One drawback of the clients I've tried is that they provide little in the way of accessibility. I like to change window sizes and enlarge the text and graphics on my screen. This is easy to do in most browsers. The clients I tried, however, all use Adobe Air. If Air provides that sort of control, these clients haven't figured out how to make use of it. Or if they have, they haven't made it clear to me how I can use it. For that matter, the same criticism pertains to Adobe FrameMaker, which uses Air as the delivery environment for its online help.

What Twitter Is and Isn't

Twitter enables people to share facts, opinions, news, and commentary about everyday life or extraordinary events.

Recently Scott Williams posted an article at http://bigisthenewsmall.com/?p=2170 called Seven Things Twitter Is Not. He makes the point that Twitter is like a river. You can stand on the bank and watch the tweets go by. They are not piling up in your mailbox. Nobody expects you to answer them. If you do toss a tweet into the river, it flows away quickly. Someone may retweet it or reply to it, but the reply floats away too. It may include some context, but 140 characters soon become inadequate to contain an ongoing chat.

Another interesting metaphor is that of a great collective consciousness. As you watch the river, a subject may suddenly attract your attention. For example, on November 26, 2008, reports about the terrorist attacks in Mumbai spiked suddenly on Twitter, drawing worldwide attention before the news media could react.

One frequently asked question about Twitter is, "How can I use it to make money?" Twitter does not seem to have answered that question for itself, but many businesses have found ways to integrate Twitter into their ongoing relationships with actual and prospective customers. The Twitter Book devotes a substantial section to the do's and don'ts of using Twitter for business. No business should jump into Twitter until its management understands the principles that O'Reilly and Milstein lay out there.

You may not understand Twitter at first, but if you try it patiently for a while and get a sense of how others are using it, it will grow on you. Go to twitter.com and give it a try.

Monday, June 1, 2009

Software Architects

This article appears in slightly different form in the May/June 2009 issue of
IEEE Micro © 2009 IEEE.



This time I look at a collection that, according to its editor, is completely different from any other book you've read. That statement is true of any book, I suppose, but this book is unusual.

97 Things Every Software Architect Should Know: Collective Wisdom from the Experts edited by Richard Monson-Haefel (O'Reilly, Sebastopol CA, 2009, 218pp, www.oreilly.com, ISBN 978-0-596-52269-8, $34.99)

Richard Monson-Haefel is a co-author of the O'Reilly books Enterprise Javabeans and Java Message Service. The project grew out of a presentation called 10 Things Every Software Architect Should Know, which he was planning for a No Fluff, Just Stuff symposium. He solicited inputs on a mailing list, and nearly 50 people contributed the "things" that became this book. Each fills a pair of facing pages and includes a contributor photo and brief biography. The regularity of the structure ends there.

The free format allows contributors to formulate their wisdom in whatever ways work best for them. They address a variety of subjects in a variety of ways, but looking at the book as a whole, I see themes emerge. They have very little to say about project retrospectives or managing risk, so I hope they will all read two excellent books from Dorset House. Norman Kerth's Project Retrospectives (Micro Review, May/June 2001) and DeMarco & Lister's Waltzing With Bears (Micro Review Jul/Aug 2003) are essential reading for any software architect.

Communication

Communication receives a lot of attention in this book. Udi Dahan of Microsoft, for example, advises you to stand when you speak to a group. It makes people take you more seriously.

Other contributors stress the importance of knowing both the language of the business decision makers and the language of the software developers. Communication failures between the two groups cause many project problems, and these contributors assign to architects the responsibility for making that conversation work.

Norman Carnovale of Lockheed Martin Professional Services emphasizes the importance negotiating with the business decision makers to ensure that mid-project changes don't lead to unrealistic schedules and ultimately to failure. David Muirhead of the Blue River Systems Group recognizes that the business must drive a project, so he advocates developing the kinds of information that business leaders can use to make good decisions. Yi Zhou lays out the elements that make up a strong business case for a proposed expenditure.

Michael Nygard reminds us that we should remember the difference between a negotiation and an engineering discussion. When business decision makers ask "do you really need x," the answer should be "yes" if that's what the analysis led to. In an engineering discussion of the point, you might say "you could do without it if . . ." but business decision makers won't hear the "if" or anything that follows it.

Another communication theme is that architects must manage their own attitudes so that they can engage in open conversations with developers and customers. They must not let overconfidence or arrogance prevent them from hearing the good ideas of others or listening to the specific needs of customers.

Timothy High of Sakonnet Technologies gives advice that is easy to follow but often ignored. He advocates recording the rationale for decisions. He questions the value of most design documentation because it is hard to keep up to date. The rationale for decisions, on the other hand, doesn't require changes and is a valuable resource for the life of the project.

Requirements

Another theme of the book is requirements. Understanding, examining, and negotiating requirements and turning them into specifications is one of an architect's key jobs. Everyone understands that this process is a negotiation that must balance the needs of customers, funders, and developers. Eben Hewitt puts forth the beautiful metaphor of a consommé -- an extremely clarified broth achieved by repeatedly straining out the solids that cloud it. You must test the language of the requirements again and again until it is clear. This reminds me of the Mary Had a Little Lamb heuristic in Gause & Weinberg's Exploring Requirements (Micro Review, Sept/Oct 2001), another book that all software architects should read.

Many of the contributions boil down to pushing back against stated requirements in various ways. Einar Landre of StatoilHydro advises seeking the value in the requirement. He tells the story of the F-16 Falcon aircraft for which an initial requirement was speeds of Mach 2 to 2.5. By asking why, the contractor was able to change the requirement to an agile aircraft that can accelerate quickly to escape from combat. Similarly, Eben Hewitt tells you not to rush to solve the problem. First interrogate it to see if you can change it into a more tractable one.

Keith Braithwaite of Zuhlke exhorts us to quantify requirements. Fast, responsive, and extensible, for example, are terms that mean different things to different people. Developments based on such terms can lead to serious disputes at acceptance time.

Assumptions, as we all know, can be dangerous. Timothy High advises us to challenge assumptions, especially our own. Urban myths -- or even facts that used to be true -- can lead us away from excellent solutions to design problems.

Not all requirements can or should be satisfied. Good design involves tradeoffs. Mark Richards of Collaborative Consulting recounts the story of the Vasa, an early 17th century Swedish ship that satisfied all of its requirements and sank the first time it fired its guns. Bill de hÓra of NewBay Software says that requirements often come in threes (like fast, cheap, and accurate) and that you should be prepared to pick two and jettison the third. When a requirement seems absolute, be skeptical and ask why. Dave Quick of Thoughtful Arts puts it another way: Scope is the enemy of success. The greater the scope, the worse the chances of success.

Another way to test requirements, according to Stephen Jones, is to stretch key dimensions to see what breaks. This kind of stress testing can identify future problems long before they occur. It enables the necessary changes before the design becomes set in stone.

A number of contributors show concern for end users. Your customer may not be the end user, as Eben Hewitt points out, but your customer won't succeed if the system serves end users poorly. Keith Braithwaite, following Heidegger, uses the german term zuhanden to suggest that systems should be invisible to users. However, Steve Talbott in Devices of the Soul (Micro Review, July/Aug 2007) takes the opposite point of view: Because it is so important to remain conscious of the assumptions and unseen factors that affect us, it's a good idea not to forget about our tools. An invisible tool still embodies the ideals and assumptions of its creator.

Pitfalls

Many contributors offer advice about pitfalls and how to avoid them. Avoiding unnecessary complexity or generalization is a common theme. Neal Ford of ThoughtWorks defines accidental complexity as the complications that arise as byproducts of addressing the essential complexity of the problem. Often accidental complexity comes from adopting frameworks that are too general or fail to match the problem.

Kevlin Henney puts it another way: simplicity before generality, use before reuse. Eric Hawthorne says to choose frameworks that stick to their own domain and don't impose unnecessary requirements. Keith Braithwaite points out that the real world has multiple, inconsistent, overlapping frameworks. Sometimes it works to use each in its own domain. Or as Randy Stafford of Oracle puts it, there is no one-size-fits-all solution. Exercise contextual sense to understand what works in a given domain.

The editor of the book, Richard Monson-Haefel, contributes his own pitfall. Avoid trying to future-proof the system, because you can't predict the future. Trying to guess the technologies of the future can lead to analysis paralysis.

Design and Development Approaches

As you would expect in a book about architecture, design and development approaches receive a lot of attention. Most contributors implicitly or explicitly advocate iterative and incremental processes. Bill de hÓra summarizes this view by saying that great software is grown, not built. Similarly, David Bartlett advocates Martin Fowler's continuous integration (CI) approach, which includes automated builds and testing at frequent intervals.

Dan Chak of CourseAdvisor, however, focuses on designing the underlying database. He feels that the data structures and relationships that underlie applications do not evolve rapidly and require a complex, comprehensive technical design up front. The resulting solid and self-protective data fortress will save application data from the inevitable bugs in the application level.

George Malamidis of TrafficBroker adopts the language of business. He says that every architectural decision is an investment. You should consider the return on investment (ROI). This is true whether you use an agile process or the waterfall model.

George Holpe of Google says welcome to the real world. He implies that most software engineers are clinging to a bygone world of predictably ordered processes controlled by the program rather than by external events. In that world, systems are tightly, rather than loosely, coupled. He quickly tears down that straw man, using the word "scary" a couple of times to characterize the new environment. I get his point, and he's certainly right about the need for distributed, loosely coupled, event-driven systems, but I'd like to point out that we were building event-driven systems in the 1960s, so the idea isn't all that new.

Several other contributors also use uncertainty to their advantage. Kelvin Hanley suggests than when you come to a fork in the road, design so that choosing between the two branches has fewer consequences. Erik Doernenberg of ThoughtWorks suggests trying both branches and deferring the choice until it's obvious which is better.

Performance and Maintenance

Most of the life of a successful software project happens after its initial installation. Many of the contributors address issues about performance and maintenance.

Randy Stafford points out that application architecture determines performance. Rebecca Parsons of ThoughtWorks advocates beginning performance testing early in the development process. This can help you identify problems early. It also gives you an ongoing record that shows which changes helped or hurt performance. Craig Russell of Sun Microsystems takes a broader view and advocates looking at the performance of the developers implementing the design.

Tools and Techniques

I don't have room to go into all of the suggestions about tools and techniques. One that tickled me is Paul Homer's suggestion that you can get a good idea of how a system works by looking at its underlying data structures. In 1967 I attended Butler Lampson's course on operating systems at UC Berkeley. Lampson made exactly that point.

Architects

Many contributions address the necessary qualities or skills of software architects. Many see themselves as leaders of the development project and focus their comments on how to interact with developers. Many say to lead by example or to keep a hand in the development work. The objectives are to maintain the respect of the developers and to experience the design from the point of view of a developer. It's easier to recognize design mistakes if you're the one who has to implement them.

Many stress the importance of maintaining respectful and open relationships with developers. Evan Cofsky of Virgin Charter invokes a taxonomy in Neal Stephenson's Cryptonomicon. Architects are kings and must provide opportunities for elves, dwarves, and wizards.

Many also look at the social or ethical aspects of an architect's work. Barry Hawkins advises valuing stewardship over showmanship. As he points out, you are playing with other people's money. Michael Nygard points out that many decisions restrict or enable end user behavior. A few days of programming time may save end users a few seconds each time they use a feature. With many users and many repetitions, that could be a huge savings for them, but at a cost that you won't recoup. It certainly raises an ethical question.

Yi Zhou tells architects to take responsibility for their decisions. That means putting it in writing and communicating it to everybody it affects. You must also follow through to ensure that developers implement your decision, and you must review the decision periodically to see how it's working out. Taking responsibility also means not making decisions that you're not qualified to make. Delegate those decisions to experts.

I hope this review gives you an idea of the flavor of the book. If you design software, you're sure to find enough nuggets of information in it to justify its price many times over.