Some data are better.

We've written about the need to repeat measurements in order to determine the uncertainty in an estimate.

Today I want to address a related question: What is the best estimate?

Let's say I want to know how high my desk is. So, I take a tape measure, and measure the desk ten times. The best estimate is the average of those ten measurements.

Now let's say that I ask one hundred people to measure the height of the desk ten times each. Now I have one thousand measurements (one hundred people, times ten measurements per person) of the height of the desk. What is the best estimate? Is it the average of those one thousand measurements?

No, it is not.

Because, someone whose ten measurements vary more than average, is probably a sloppy measurer; their results should be understood to have more uncertainty than other peoples' measurements, and should be somehow discounted.

And, someone whose measurements are significantly different from the group average, is probably an odd measurer – maybe their problem is optical parallax, or not knowing how to use a tape measure – so their results should be somehow discounted.

This insight – that some data are better than others – is what allowed Drs. Joern Diedrichsen & Reza Shadmehr to develop an approach to improve brain activation estimates from functional MRI data that contain artifacts.