voodoo baloney & a dead fish

We've written here about voodoo baloney (that's what happens when you use some criterion to select data, and then perform, on those selected data, a statistical test, related to that selection criterion, which assumes that the data are completely random, rather than pre-selected).

A related problem concerns multiple comparisons: If you try so many different things that one of them is bound to turn out right, just at random, but then pretend that the one that turned out right was the only one you tried, you're fibbing. (In the context of functional neuroimaging, those "many different things" are the many different locations in the brain.) To illustrate this point, at the Human Brain Mapping meeting in San Francisco this week, there will be a presentation on functional brain imaging of a dead fish.