Quality Control in Computers and Publications
by Kim Iles

It’s just a never-ending battle, and the ground it is fought upon keeps shifting. 

Computer programs were always a problem, and the code never seemed to be checked by the people who actually understood the details.  In some cases, they were not allowed to – only the geeks had access.  It helps to have a standard set of data around that has been run on what you are sure produced a reliable output (and perhaps was checked by hand).  When a new program or revision comes out, you can rerun the data and perhaps detect changes or problems.  On the other hand, there are always exceptions and odd combinations of things that are not caught by a particular data set.  When you cannot view the computer code in proprietary software, this is about all you can do.  “User Groups” are a great thing, and you ought to keep in touch with them.

There is a good reason that computer products come in “versions”.  If you have version 2.0 and there is a version 2.1, you have to wonder not only what has been added, but what might have been corrected (or corrupted).  Programmers do not always keep complete and accurate records of what has been changed from past versions.  This is another instance when an example set of data and a copy of the former printout is useful.

EXCEL poses a new problem.  I really like EXCEL because it is self‑documenting and pretty simple to read (if you don’t get into fancy formatting and tricky tables).  You can also use the “audit functions” to show you where data comes from or goes to.  It’s also easy to change an example and see what the result would be.  If authors of statistical papers made available a simple EXCEL program to illustrate their difficult notation the world would be a better place.  One of the problems is that a formula or constant can get corrupted or changed without you being aware of it.  Here again, the audit functions can help to make sure that does not happen, but quality control is a constant issue. 

I was once caught in this myself, with a program for computing Relascope scale BAF’s.  Not having an American Scale Relascope (I prefer the Wide Scale) I did not check it physically.  Sure enough, a constant was somehow corrupted in the earliest version.  I did my best to contact people, and corrected all future versions, but an unintended keystroke or copy command is a constant worry.  Even when you have the original disc or program, you are never sure what copies were made and stored in another place.  All you can do is to warn people to verify that they have the most current version of a program before they use it for any important purposes.

When borderline trees are checked you should immediately find an error in the BAF from a prism or Relascope (I have known prisms to be marked incorrectly at the factory), and I recently tested an electronic Relascope that was calibrated incorrectly from the time of purchase.  There is just no substitute for testing.  I once found a target used to calibrate prisms that had been wrong for 20 years.  It showed up when a Relascope failed for that BAF, but was correct for others.  I doubt that it had ever caused a problem, because that BAF was a rare one in the Northwest.

Most textbooks, even in the 6^th or 7^th edition, still have a long list of typos.  Do you ever see these corrected on the pages of the texts?  No - and the typo list is lost long before the book stops being used.  I have seen reports introduced in court that had errors of a critical nature – even those done by experts in the field.  Microsoft Word changes fonts every once in a while, and you can never be quite sure what symbol might be brought into an old copy of a Word Document.  What gets displayed on another computer is never certain either, since MSWord will sometimes substitute fonts if they are not available on that computer.  Luckily, “pdf format”, solves some of these problems.

Corrections to printed papers are still a difficulty.  As George Furnival once said “the corrections to a paper never seem to run fast enough to catch up to the original publication”.  Older published papers are a bit easier to check, since you can examine the “errata” sections of the following few issues for reported errors by the authors.  Some errors, however, and not caught until years later (and some are never reported at all).  The best thing is always to put through a quick phone call to the author of a paper, procedure, or computer program to check on the current status.  Authors are easy to locate on the Internet.  You can often get updates that way and improvements to the original.  It’s actually appreciated by most authors, who seldom know that anyone is even reading their papers, and it’s a good way to make contacts.

The printing process used to be a problem.  When lead type was used, the printers often inserted errors by mistake.  The author was not always told of changes made in the editing or printing process (and that is still true today).  Even when authors see the final product those changes are not always pointed out and might be missed.  Hard to believe, but true.  As Lew Grosenbaugh once stamped onto reprints he sent out from his article “The Gains From Sample‑Tree Selection With Unequal Probabilities” :

“These 4 pages should replace garbled pages 203‑206 in the March 1967 Journal of Forestry, where printer ignored type-setting errors indicated in galley‑proof while garbling text set correctly in galley-proof.”

People who read the original journal article would never have known this unless they checked with Lew personally or asked for a reprint.  You would not believe how little control authors have over their own textbooks when they go through a publisher.  Copy editors have a very heavy hand.  Although they deny ever changing an authors meaning, it does happen.  The printing process is less complex these days, but it is also more hurried.  I don’t suspect the error rate has diminished.

All of this reinforces the issue of checking everything we do in the woods - every procedure, every instrument, every program - and with check cruises to verify the data.  A great many field procedures in natural resource management have no auditing procedures at all.  In this respect, forest inventory is way ahead of many fields.

As I once told a Psychology class taking statistics ... 

“I know you are taught how to treat people who are paranoid, obsessive‑compulsive and fixated on triviality … but don’t cure them.  Send them to me and I will put them on field crews. 

        They are worth their weight in gold.”