Questions from the Field:
What exactly IS “bias” anyway?
Well, part of the confusion comes from the fact that the common interpretation of the word “bias” implies a slanted viewpoint or intention. The “bias” we are talking about is the statistical term. It has nothing to do with intent. It is about a potential outcome from a repeated sampling procedure.
In statistics, when the average of all the possible outcomes is the actual answer for the population, we call that an “unbiased” method. If it does not come out to that exact answer, the difference is the “bias” for that method. Some of the sample answers may be too large, and others too small, but they are KNOWN to cancel each other for an unbiased system. You get this kind of unbiased system by following some well known procedures. Both random and systematic sampling methods are normally unbiased (at least as far as they can be – this assumes that the compilation, measurements, etc., are also correct). An unbiased system does not give the right answer every time (that is what sampling error measures) – but it is right on the average.
Unbiased methods that are “helped” are often turned into biased systems. You may do it with the best of intentions. You might not like, for instance, the random pattern that is generated by a sampling system – so you discard it and choose another sample (meaning well, of course, and trying to do a good thing). The chance of that process remaining unbiased is almost zero. You might not like the many border plots that are generated by a systematic sample, so you shift the grid around to make it look better. Bias rears it ugly head again. The plot actually lands in a patch of blackberry vines. The trees look just about the same 20 feet away in a nicer spot … you get the picture. This can be just as biased as changing the recorded diameters of trees on a plot because you do not “believe” that they are this big in the rest of the stand. Even if you are right, and have landed in a patch of large trees, moving the plot or changing the measurements is biased. It is the process we are talking about here [1].
None of this is about intent. It is not about fraud. It is about a long‑run result. “Meaning well” and not knowing (or caring) what change is caused does not mean that your methods are unbiased. Even if it did give a better answer on that plot (or stand) you may very well be causing a bias. It is possible that actions which generally give better answers with individual small stands may give much worse answers when those stands are combined into larger areas.
“Unbiased” has a fairly specific meaning in statistics, and to keep something unbiased you sometimes have to keep a sample that does not appear comfortable to you. Otherwise you are “causing a bias in a good cause”. If you are risking your own money, go right ahead – but you would not want to get called into court. You should at least be pretty careful about doing these adjustments, and we might suggest that you not do it at all unless you know EXACTLY what you are doing and are prepared to explain the risks.
[1] If these things are such exceptions, why is it that people keep finding them on plots all the time?
Have you ever done anything to prove that your judgment was right about this kind of adjustment?
Originally published August, 2005