Why not document?

Four myths about documenting software development

Let’s talk about insurance, and how important it is to business.

Most companies have it. Their policies cover worker injury, fire, litigation, Acts of God, defamation, interruption of business, et cetera, et cetera.

So why are companies so unwilling to enforce documentation policies?

Documentation is crisis insurance. Small companies think they can get away without it, large companies are afraid of disrupting the status quo. Neither can afford not to have it, but they try. And they fail.

One of my personal long-standing tests of software engineering is the Bus Test. Every manager should give himself the Bus Test at least once a month. A really good manager will give himself the test weekly. Superb managers give it to themselves constantly.

What is the Bus Test? It consists of asking this question: “If John Q. Programmer were to be hit by a bus tomorrow, how long would it take to train his replacement?” Then ask it about a critical subsystem group. The true acid test is: If the whole project team were to meet the business end of a bus, how long would it delay the project?

When I was fresh out of Berkeley, I brought up the Bus Test as many times as possible in every conversation I had. Most people thought I was joking, so I changed it to the Resignation Test (“If all your programmers got better offers from Sun…”). Then, during my tenure at ASCNet, lo and behold, one of the programmers did indeed get in a car wreck. (Needless to say, many joked that I had gone too far to prove a point.) As you might have guessed, we failed the test. It was only through the heroic efforts of a few programmers (and that the injured programmer was working on a non-critical subsystem) that the crisis did not escalate into a full-blown catastrophe. (But it did incur hidden costs.)

Myth #1: Documentation is too time-consuming

This is the whine I hear from large companies: their programmers are so busy that they don’t have time to document. This is, and I’ll put it bluntly, bullshit. For one thing, the programmers might not be so “busy” if they had proper documentation from previous iterations of the design cycle. Not only would they be spending less time tracking down the authors of legacy code, but they might also be able to use code from previous iterations and depreciated experiments instead of reinventing the wheel. If you plan to establish a reusability process, you better have a documentation process nailed down first.

If your programmers and engineers really are that busy, then you might consider hiring on some technical writers and a documentation librarian as additions to your engineering team. Technical writers pay for themselves within a few design iterations, while librarians have a much bigger impact farther down the road.

Which brings us to…

Myth #2: Documentation has a poor cost/benefit ratio

This is the bleats from small companies and consulting firms; the former being strapped for cash and the latter being bloodsucking vampires. Since documentation is a type of insurance, benefits cannot be quantified easily. Small companies seem to suffer from American Corporate Myopia even worse than their larger counterparts, and are quick to discount the need for documentation. “It’s a luxury we can’t afford,” one company chief remarked to me. “Time and money disallow it.” Yet these same companies often pay for insurance above and beyond the mandatory minimum. Strange.

Consulting firms have a better excuse: it doesn’t benefit them. Which is true, to a point. Documentation only benefits the customer and, in many cases, alleviates the need for the consulting firm’s continued presence. But consultants live and die by reputation, and if you get a reputation for “creating job security,” as one consultant put it, you’ll quickly find out just how smart your customers really are.

Myth #3: Programmers and engineers have poor communication skills

This myth is almost justifiable, considering some of the documents I have read that were written by fellow engineers. Pity most technical schools don’t require extensive coursework in communications; if nothing more, such a requirement could improve the quality of written user manuals.

Again, the solution is to hire professional technical writers. If that is too costly, make a hiring requirement be that the person has above-average communications skills. Explain to them that part of their performance appraisal will be on the quality (not quantity) of their written work. Leave the programmers with poor communications skills to the large companies that can afford to bury them in the bureaucracy.

Myth #4: Nobody reads documentation anyway

I do. And I’m certainly not nobody.

The Counter-Myth

Now I’m going to prove that I’m a complete hypocrite. I still believe in the above tenets, but…

The company with whom I worked in 2010 has a documentation policy, but it is documentation for documentation’s sake — there’s no verification of the finished products, and the finished products are both redundant and completely unusable.

Modern tools like Javadoc (and its C++ equivalent, Doxygen) eliminate the need for detailed API specifications, because the specification is near the code itself. This is incredibly convenient for the developer, and encourages the synchronization of code and documentation: when a change is made, the code review will show if the developer updated the documentation when he updated the code.

The problem lies with our director of operations. He is uncomfortable using UNIX (in this case, Linux); in fact, I would be surprised if he could even use UNIX at all. He wants all the documentation to be provided in — wait for it — Microsoft Word format, so he can read it.

Thus the documentation for the API is far from the code itself: on a completely different operating system, in fact. This lack of locality means that the documentation usually is updated en masse and often not updated at all. So this inspires…

Myth #5: Any documentation is good documentation

In short, do not introduce arbitrary gymnastics in the documentation policies. Remember that the primary consumers of internal documentation are developers and technical writers; “management friendly” internal technical documentation is counterproductive, quickly becomes obsolete, and is worse than no documentation at all.

1The “phones-per-order” debate was a nasty one at ASCNet. During a crisis, one of the programmers had completely revamped the phone order software, without consulting the rest of the team for ideas. While the software was much more functional than its predecessor (especially considering the time taken to write it), there was one glaring deficiency that seemed to rattle everyone but that programmer.

First, some background: ASCNet was in the business of personal emergency roadside cellular phones. Not only did we run the support service, we also manufactured and sold the phones. To control the “huge” number of orders we received, we wrote some software to take orders and forward the appropriate information to Billing and Manufacturing.

The problem was: How does one define an order? The programmer defined an order as a phone; therefore, if you purchased multiple phones, you were assigned multiple order numbers.

I had two arguments against this: one, it made life difficult for the consumer, who was expected to remember that ‘Q1624-3525’ was Aunt Polly in Philadelphia; and two, it did not take into account the (admittedly rare) possibility that a customer would place an order without purchasing a phone (say, for example, to order accessories).

The rest of the team agreed wholeheartedly — Jim Earhart was even heard to exclaim “Give that man a raise!” — but the programmer was adamant. Unfortunately, it was a fait accompli: the software was already deployed, and we had more pressing matters of concern.

This still irks me to this day…

2American Corporate Myopia: The inability to see past six months when computing profitability, making business plans, et cetera.

3Large consulting firms like Anderson and EDS often get away with this crap; look for a future scathing editorial on them. [This was written in 1998, back when few people knew just how unethical Anderson was — rgm]