This time I look at two subjects that confuse a lot of people and give rise to a great deal of wishful thinking. Misconceptions about software testing can cause great damage to large and small enterprises. Misconceptions about freelancing can be catastrophic for individual entrepreneurs.
Software testing
Perfect Software and Other Illusions About Testing by Gerald Weinberg (Dorset House, New York NY, 2008, 198pp, www.dorsethouse.com, ISBN 978-0-932633-69-9, $29.95)
Testing is a sampling process designed to obtain information to use in decision making. Some testing activities entail banging on a keyboard, but many others do not. Suggesting a design review, for example, is a form of testing and a developer's reason for not wishing to submit to a review can be a useful test result. Many people see testing purely from the banging on keys point of view. That and other misconceptions lead to painful consequences for them. Gerald Weinberg, a self-described empathic person, has written this book to ease their pain.
Weinberg has been in this business a long time. He was an old timer in 1973, when I first read his enormously influential Psychology of Computer Programming, and he has been an effective and productive writer and consultant in the field of developing and testing software ever since. I have reviewed several of his books in my columns over the years. In addition, many books by other authors, especially those published by Dorset House, show the profound influence of Weinberg's style and approach.
The insights of the late Virginia Satir, a family therapist, inform this book, as they inform much of Weinberg's work about the social aspects of managing software development. Weinberg applies Satir's decomposition of communication into intake, meaning, significance, and response to structure his lessons about how to use testing. He also uses her insights about defensiveness and survival rules to understand the natural immunity to information that people often experience in threatening situations. Nonetheless, there is no new-age mumbo jumbo in this book. Weinberg translates all of his psychological insights into practical terms.
The book's title fallacy concerns the widely held belief among the general public that companies could produce bug-free software if only they tested everything before selling it. This belief falls apart upon cursory examination. Testing "everything" is not possible in a finite amount of time. Furthermore, while testing can identify problems, it cannot correct them. Testing can only provide information. Those in charge of a project can use that information to help them assess the risks associated with various options -- such as shipping the product or sending it back to the developers for more work. This decision is not always easy, because fixing one problem often leads to introducing or uncovering another, sometimes more serious problem.
Because testing can only provide information, you should test only if you need information to help you make a decision. Furthermore, written test reports rarely tell everything that the testers know, so if you want your money's worth from testing, you need to question and interpret the results. It also follows that testers do not produce decisions. Bad managers ask testers if the product is ready to ship. Good managers ask testers what they found, then use that and other information to help them make their own decisions.
Weinberg has seen the insides of far more projects than most of us have and has worked with some relatively dysfunctional organizations. His stories of scapegoating, intimidation, and other misuse of testers are hair raising. If you work under such conditions, you need to read this book right away.
Weinberg seems to have developed a special style just for this book. His chapters average about nine pages. Each ends with a short summary and a long list of common mistakes. The mistakes section reiterates the earlier chapter material in negative form. For example, his chapter on information intake explains why you should choose which information to look for among the large amount of raw data available to you. The first common mistake at the end of the chapter is "Not thinking about what information you're after. Don't just take what comes. You have choices."
As in many of his other books, Weinberg presents some lessons in the form of vignettes. For example, he builds his chapter about major testing fallacies around a few hours in the life of Rose, the new testing manager. The dialog is contrived, and the realistic touches (Rose spears a pickle with her fork and waits for Barry to finish his chili) are lame. Nonetheless, Weinberg uses this form effectively. He shows how the pressures on the different players combine with their partial understanding (or misunderstanding) of the situation to lead to scapegoating, turf battles, and finger pointing.
Weinberg presents a breakdown of bug fixing that accounts for a lot of inefficiency and delay in software development projects. Testing can uncover an anomaly, after which further testing merges into another phase called pinpointing, or isolating the conditions under which the anomaly occurs. Failure to agree on who is responsible for pinpointing leads to many conflicts between testers and developers.
Pinpointing leads to locating, which means determining the section of code in which the problem resides. Few organizations regard locating as part of testing, but a desperate developer might be tempted to return the bug to the tester for "more information."
Between locating and fixing comes another important step: determining significance. This is a management function belonging to neither testers nor developers. Many stakeholders can contribute to this process. Ultimately, someone must weigh the costs of both alternatives. Fixing the bug takes time and effort and might break something else. Leaving it as is can result in other costs, such as service calls or lost sales.
One of the biggest problems associated with testing is how it interacts with the schedule. In an ideal development process, testing begins before the project, because deciding whether on not to do a project depends upon information gleaned from some form of testing. Then testing continues through all phases of the project. Unfortunately, most project managers follow a different model.
Typical projects have schedules that include a block of time labeled testing at the end. This block does double duty as a buffer to accommodate slips in the development schedule. In a good project, testers participate in reviewing the usual project artifacts -- market requirement documents, functional specs, user interface specs, detailed design specs, and documentation plans. In a bad project they may be handed a functional spec and told to develop a test plan that tests everything. In either case, unless they work alongside the developers throughout the project, they do not usually find problems until long after those problems enter the product. What could have been fixed easily months earlier now becomes an obstacle to delivery. A project that was "on schedule" until the testers got hold of it is now in danger of slipping. At that point, it's easy to blame the problems on testing.
Another distinction that Weinberg makes is between testing and demonstrating. If you know the "test" result in advance and are only running it to give potential customers a good feeling, you are not testing. Testing always seeks to find new information. Somewhere near the level of Gianni Schicchi in Weinberg's Inferno are the sellers of expensive testing systems who use such fake tests to support unsupportable claims about their products. Vendors who claim that their products will do all the work without receiving extensive explanations about what the programs they are testing are supposed to achieve are simply charlatans in Weinberg's view. Weinberg is firm in his belief that machines will never replace brains when it comes to software testing. Testing is an interactive process of investigation in which each finding leads to new questions.
Like all of Weinberg's books, every page of this one has wonderful insights and advice. If you have anything to do with software development, you should get a copy and read it at least twice.
Freelancing
The Principles of Successful Freelancing by Miles Burke (Sitepoint, Collingwood Australia, 2008, 206pp, www.sitepoint.com, ISBN 978-0-9804552-4-3, $39.95)
SitePoint is a small publisher targeting web professionals. Its books deal with SQL, CSS, HTML, ASP.NET, and similar staples of the browser-based world. Miles Burke is a freelance web developer, and he has written a book for others who wish to enter that arena.
I confess that I was put off by the lax editing and the Australian jargon of this book, but if you can get past that, it has a lot of useful content for people who want to set out on their own. Other books for freelancers (for example, Secrets of Consulting by Gerald Weinberg) focus on advanced topics. This book lays out the basics. If you read this book carefully, you won't make any major omissions as you prepare for your freelance career. In fact, at every stage, Burke puts on the brakes and urges you to move slowly.
Many people jump into freelancing because they have just lost their jobs. Such an approach has little chance of success in most cases, because starting a business requires a financial cushion that few recently fired workers have. Also, those who have never worked for themselves may overlook expenses like self-employment taxes, insurance, sick days, vacations, and the like. Burke shows you how to make the necessary computations and set your rates accordingly. You may have to adjust a little if you don't live in Australia, but many of the points he makes apply in the USA as well.
Burke covers all the bases -- from not using stolen software to negotiating with your family for an efficient work area in your home. He also gives good advice about how to find work, how to relate to clients, and how to balance life and work when they all happen in the same place.
If you are considering a career as a freelancer, especially in the web design area, you should find this book well worth its price.
