Owen Yang
I have mentioned them a few times perviously but feel it could be beneficial to write a few sentences here.
P value in a nutshell
A P value is possibly best described as the chance that an observation is a reasonable sample from a reference population, or the chance that a group of observations are a reasonable sample from the reference population. When P is very small, i.e. this chance is small, we would think these observations might not be from that reference population. This is when we say these observations are significantly different from the reference population.
If the reference population is the blood pressure in placebo-treated patients, and the observations are the blood pressures in medication-treated patients, then then p value of 0.05 means the chance is 5% that medication-treated patients are just a random sample of placebo-treated patients.
What the statisticians technically would like to say is that the chance is 5% that medication-treated patients are just a random sample of placebo-treated patients when the blood pressure in the medication-treated patients is more extreme than the observed value. So if the mean blood pressure is 150 in medication-treated patients and 160 in the placebo-treated patients, 5% is the chance that a blood pressure of 150 or more extreme (in this case 150 or less) can be observed when the medication-treated patients are in fact just a random sample of placebo-treated patients.
α value in a nutshell
α values, in my words, is some sort of false-positive rate, or false-difference rate. It is the rate that a difference (between two groups) can be observed when there is in fact no real difference, and the reason that the difference can be observed is because of randomness.
Although we think the observations could be different from the reference population when p<0.05, there is still a 5% chance that they are from the same population. So when in fact they are from the same population, 5% is the chance that we make the mistake and erroneously think they are different.
It is easy to think this is synonym to say that the 5% is the chance that we are wrong after we see a difference, but I am not entirely sure about this. I am more certain to say that this is the chance that we observe a difference before we actually observe the results, rather than the chance of being wrong after we have observed the results. To distinguish them seem important to statisticians, but I admit I sort of feel it okay to be soft about it in most circumstances.
The p value vs α value
There is some link between P and α value but as I am writing now, I actually do not have a good way to tell this link. Please let me know if you find a easy way of this.
