Accuracy and precision
Part of what makes uncertainty such a slippery subject is that it conflates several concepts that are better kept apart: precision, accuracy, repeatability, and reproducibility. People often mention the first two, less often the last two.
In a nutshell:
- The degree of closeness to the true value. Freedom from bias.
- The degree to which an instrument or process will repeat the same value. Freedom from noise.
- The degree to which an instrument or process will repeat the same accuracy and precision statistics over time.
- The degree to which other instruments or operators using the same process will repeat the same accuracy and precision statistics over time.
What's the difference?
It's clear that precision and accuracy are different things. If someone's shooting at you, for instance, it's better that they are inaccurate (high bias) but precise (low noise) so that every bullet whizzes exactly 1 metre over your head. But, though the idea of one-off repeatability is built in to the concept of multiple 'readings', scientists often repeat experiments and this wholesale repeatability also needs to be captured. Hence the third drawing.
One of the things I really like in Peter Copeland's book Communicating Rocks is the accuracy-precision-repeatability figure. He captured this concept very nicely, and gives a good description too. There are two weaknesses though, I think, in these classic target figures (above). First, they portray two dimensions (spatial, in this case), when really each measurement we make is on a single axis. So I tried re-drawing the figure, but on one axis:
We are blind
The second thing that bothers me is that there is an implied 'correct answer'—the middle of the target. This seems reasonable: we are trying to measure some external reality, after all. The problem is that when we make our measurements, we do not know where the middle of the target is. We are blind.
If we don't know where the bullseye is, we cannot tell the difference between precise and imprecise. But if we don't know the size of the bullseye, we also do not know how accurate we are, or how repeatable our experiments are. Both of these things are entirely relative to the nature of the target.