It is unanimously agreed that statistics depends somehow on probability. But, as to what probability is and how it is connected with statistics, there has seldom been such complete disagreement and breakdown of communication since the Tower of Babel. Savage, The Foundations of Statistics (1972)
Recentely, I had to give a presentation about P value and I discovered during the preparation of this talk a battle to the death within statistics. It is enough unique to be reported and commented.
A little bit of history of statistics, between the 18th to the 19th century, Bernoulli, Bayes, Laplace, Boole, Venn and others developed lots of ideas about probability and statistics. At this time statistical inference was Bayesian. Then it required prior probability. Sir Ronald Fisher didn’t want to specify a prior, so he constructed a new framework for statistical inference around 1920s. Part of this is what he called significance testing and significance levels, which we now call P-values. Jerzy Neyman and Egon Pearson “extended” Fisher’s ideas and wrote about hypothesis testing (1930s).
Fisher wanted a way to test the results from experiments. The motivation questions he had were: How does the data match to my hypothesis (i.e the assumed model)? But he still refused to assume a prior probability. On the contrary Neyman does not assume many repeated experiments from the same population. He wanted the researcher to have a tool to help evaluate the strength of evidence.
Fisher and Neyman couldn’t resolve the differences between their ideas. They argued for 25 years until Fisher died in 1962. Fisher proposed a way to measure how likely the results of an experiment are under some assumptions. Neyman proposed to use cut off levels to decide whether the data matched to a hypothesis, with the cut offs chosen to limit errors.