In 1968 Arthur C. Clarke and Stanley Kubrick created the film 2001, A Space Odyssey. In the January 2001 issue of National Geographic, Clarke discusses prospects for achieving in real life some of the fictional events of the film. He does not, however, discuss HAL, the murderously megalomaniacal computer that both enabled and endangered the mission. HAL seems even further from reality today than in 1968.
One of the books I look at this time quotes Berkeley computer science professor Robert Wilensky as saying that the social and psychological issues are so hard that computer scientists can only hope that they aren't on the critical path. I think this hope is wishful thinking, as HAL's continuing unreality suggests.
This time I look at books that confront the context, largely social, in which today's technological advances occur. Many technologies make great leaps forward at the surface level but never really take hold. Entrenched technologies have largely implicit and unnoticed support systems. Recognizing these systems and addressing the problems they solve is the way to achieve deep and lasting change.
The first book I look at asserts that tunnel vision leads pundits and producers alike to misjudge the impact and support needs of new technologies. It surveys the main areas in which infohype has gone badly wrong, and explains why.
The other books I look at describe methodologies for developing software-based products. The first methodology is Extreme Programming (XP), which I discussed in my review of Kent Beck's Extreme Programming Explained (Micro Review, Nov/Dec 1999). XP focuses heavily on the actual practice of programming and the realities of customer-developer communication. XP is a highly effective way for small teams to develop software incrementally.
The second methodology is Contextual Design (CD), which lies at the other end of the size spectrum. It is based on Karen Holtzblatt's Contextual Inquiry method, which places heavy emphasis on observation and interviews to find both explicit and unnoticed aspects of the customer's activity. CD assumes a stable, functioning customer process. It organizes the resources of a large development organization or corporate IT department to achieve an auditable design that relies on structured data about the customer's activity to resolve design questions.
XP is like planning and executing a drive to the store. CD is like planning and executing a voyage to Jupiter. Both methodologies, however, have ideas you can apply to projects of other sizes.
Social Context
The Social Life of Information by John Seely Brown & Paul Duguid (Harvard Business School Press, Boston MA, 2000, 332pp, ISBN 0-87584-762-5, www.hbsp.harvard.edu, $25.95)
John Seely Brown is head of the Xerox Palo Alto Research Center (PARC). Paul Duguid is a researcher at UC Berkeley. He specializes in social and cultural issues in education. Both draw heavily on their backgrounds for examples to support the analyses in this book.
Information is a convenient construct. It gives us insight into many aspects of modern technology. Like all models, however, it ignores many details. Such simplifications can lead to great advances -- Newton's equations for planetary orbits, for example. This only works when the ignored details are insignificant to the problem at hand.
Unenlightened or unscrupulous futurists, business consultants, and product developers have applied the information construct too broadly:
- Books are information containers.
- Conversation is information interchange.
- Learning is information absorption.
- Organizations are information consolidators.
- Office work is information handling.
- Business processes are information flows.
These and many similar oversimplifications all contain grains of truth. Ideas based on them, however, have fallen far short of delivering their promised benefits. For example, the reengineering movement of the mid 1990s succeeded for well defined processes like procurement or shipping. It failed for fuzzier, less infocentric tasks like insurance claim processing. Such tasks depend on more than just a formal process for moving disembodied information. They have social components: the mutual support and shared knowledge of specific human beings.
In case after case the authors return to the same theme: from product development to company reorganizations, innovations fail when they ignore or try to suppress the social support systems that made the pre-innovation situation work. And sometimes innovations succeed only because people find ways to sneak the support systems back into the picture. This leads to a valuable clue to making struggling systems succeed: pay attention to what won't budge. If it's important to the people using the system, include it in the system. Don't try to stamp it out. Reinforce it instead.
The authors cite an excellent example of this. Dispatchers of field support reps got feedback from the dispatched reps when the reps called in for their next assignments. After a while the dispatchers became skilled at analyzing customer problems and only sending the field reps when necessary. When the company adopted a new system that didn't require the reps to call in, new dispatchers were less effective -- except for one who happened to sit near an old-timer and managed to learn from her. By recognizing and fostering this support system, the company helped the new system to succeed.
The authors summarize their theories of education and learning. They have obviously thought a lot about this subject, but they decided not to elaborate in this book. Their presentation is concise, but thought provoking. The essence of it is to identify the core competences of a university. If these were simply variations on causing students to absorb information, universities would indeed be threatened by the proliferation of online courses. Instead, the authors identify the social factors that have made universities such enduring institutions.
I especially like the fact that they identify misrepresentation as one of the core competences of a university. What they mean by this is that a degree from a good university bundles in a certain amount of experimentation on the student's part that probably could not stand on its own if unbundled and offered as evidence of the student's qualification for a specific job. It is a cross-subsidy of an essential part of becoming an educated person.
One of the most interesting parts of the book deals with the relationship between organizations and the collective knowledge of their employees. On the one hand, many organizations have difficulty accessing and using their knowledge. The perfect example of that problem is the difficulty Xerox had in applying the GUI technology that their employees at PARC developed. Xerox owned the knowledge, but it could not use it.
On the other hand, knowledge flows into and out of organizations like water -- regardless of intellectual property laws. Networks of practice link employees in different firms, especially in concentrated areas like Silicon Valley. The same example applies here: the folks at Apple had no difficulty understanding, and eventually exploiting the Xerox PARC GUI work. Furthermore, because knowledge flowed more freely within Apple than it did within Xerox, Apple was able to bring the GUI to market.
The authors also discuss office work and the way some analysts ignore its social aspects. Anyone who has accessed a company network from off-site equipment -- in a home office, for example -- rediscovers the value of division of labor. Functioning as your own system administrator and IT department is an expensive and not very valuable exercise in reinventing the wheel. Also, incidental learning is harder to come by and shared knowledge harder to access from outside the traditional office site. By ignoring the invisible role of social systems, an infocentric view of office work fails to address and solve these problems.
The advertising agency Chiat/Day performed the ultimate experiment in stamping out the social aspects of office work. Employees had no permanent equipment or desk space. Instead they checked out communal equipment from a pool each morning, then sat wherever they could find space. Each night they returned the equipment to the pool and went home. This experiment, as you probably guessed, failed. The authors attribute the failure to Chiat/Day's management's failure to recognize and understand the value of the social aspects of office life.
The authors apply a similar analysis to agents. Viewing humans as goal pursuing agents hides the importance of the social nature of learning, taste, choosing, brokering, and negotiating. HAL can handle all of these human foibles, which is why HAL remains in the realm of fiction.
Finally, I love the analysis of printed documents and the social importance of their fixity. Among other things, the authors say
Efficient communication depends not on how much can be said but on how much can be left unsaid -- and even unread -- in the background. And a certain amount of fixity, both in material documents and in social conventions of interpretation, contributes a great deal to this sort of efficiency.
This book is full of common sense. It deserves to become a strong and beneficial influence on the way we think about designing software and processes.
Extreme Context
Planning Extreme Programming by Kent Beck (Addison Wesley, Boston MA, 2000, 158pp, ISBN 0-201-71091-9, www.awl.com, $29.95)
Extreme Programming Installed by Ron Jeffries, Ann Anderson & Chet Hendrickson (Addison Wesley, Boston MA, 2000, 286pp, ISBN 0-201-70842-6, www.awl.com, $29.95)
The XP community has flourished since Kent Beck's first book on the subject appeared a little over a year ago. Addison Wesley labels the two new books as part of The XP Series, and as Kent Beck points out in his foreword to Extreme Programming Installed, the fact that somebody else wrote it bodes well for the future of the discipline.
Extreme Programming Explained is full of good ideas, but very concise, and Beck's new book is the same way. Extreme Programming Installed gives a more detailed and patiently elaborated view of how to do XP.
Because I discussed XP in my Nov/Dec 1999 column, I don't go into much detail about the basics here. The key tie to the theme of this column is the fact that, as Beck says, XP practices depend on human creativity and accept human frailty. They integrate the social support and informal communication that more mechanical methodologies might ignore or try to suppress. They use index cards and a few simple wall charts rather than lengthy requirements documents, design specifications, and project tracking software.
The XP books include little actual code. Occasional examples in Smalltalk, even as simple as they are, can put off many programmers, so I'm glad that Extreme Programming Installed contains two detailed examples of how to apply XP principles in the Java environment. I think all programmers -- whether they wish to adopt XP or not -- should read the chapters XPer Tries Java and A Java Perspective. These chapters convey a sense of the importance XP assigns to development tools, and they give a remarkably clear explanation of how to let testing drive design.
XP is simple and powerful. Get these books and read all about it. Then find a group that feels the same way, hook into the XP community, and put XP to work.
Heavyweight Context
Contextual Design by Hugh Beyer and Karen Holtzblatt (Morgan-Kaufmann, San Francisco CA, 1998, 496pp, ISBN 1-55860-411-1, www.mkp.com, $44.95)
This admirably clear book patiently and thoroughly describes a methodology for gathering and using the data necessary for true customer-centered design. By contrast with XP, which adds a customer to the design team, CD starts from the premise that customers can't represent themselves in the design process. When they are outside their usual work environments, they can't explain what they do or what really matters to them. Furthermore, they don't understand the details of developing the software for their own areas, let alone the other issues that developers have to keep straight.
Whereas in XP the customer speaks with a single voice, CD assumes that there are many customers and that nobody in the customer organization can represent them. The only way to produce a system that supports all of their needs is to understand all of their needs.
For these reasons, CD begins with a research project: find out everything about how each user does his or her work. An interviewer goes to the customer's work site and for several hours acts as a combination between an apprentice to the interviewee and the interviewee's partner in documenting the work practice. The interviewer compiles and interprets the information, then goes over it with the interviewee until they reach a common understanding.
Interviewers repeat this process for each class of worker until they have a complete set of data. They then produce narrative descriptions and graphical representations of a variety of models and arrangements of the data.
A key to the process is ensuring that the developers fully understand and assimilate the results of the customer study. This becomes the data that developers turn to when they have questions or disagreements.
This summary hardly does justice to the entire CD methodology, but the book covers every aspect of it thoroughly. Following a pattern that suggests that they applied their methodology to their own process, the authors make it easy for you to survey CD at the executive summary level or to delve into the details of areas that interest you. At every level the writing is clear and the concepts are easy to understand.
This book is not for everyone. If you are an IT manager or the head of a large consulting organization, you should assign someone to investigate whether CD can help your organization understand and serve your customers.
