>"If you do a year of research and get a null result you are screwed"
The best I heard on this site was that "alpha is the expected value of p". After a few years of calibration, each field collects just enough data (or adjusts the cutoff) to pump out "statistically significant" results at the appropriate rate. It makes it just hard enough to seem like an accomplishment, but most grad students will get something to publish if they just keep at it.
Of course, that is a totally arbitrary metric that has little to do with the quality of the research.
We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist. And they still manage to produce tons of significant results and findings, despite using much higher standards than regular social sciences.
> We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist.
That this "phenomenon" does not and cannot exist is the foundational assumption of modern science. Back in the days when the Laws of Thermodynamics were being figured out to describe the properties of steam engines, a few chemists realized that everything in the universe could be described in terms of indivisible 'atoms'.
Atoms aren't indivisible anymore, but the assumption lives on.
Life is much more interesting if one assumes that our 3d-reality is but a projection of a higher-ordered reality.
The best I heard on this site was that "alpha is the expected value of p". After a few years of calibration, each field collects just enough data (or adjusts the cutoff) to pump out "statistically significant" results at the appropriate rate. It makes it just hard enough to seem like an accomplishment, but most grad students will get something to publish if they just keep at it.
Of course, that is a totally arbitrary metric that has little to do with the quality of the research.