Revision Date: 2005-09-06
Problem:
| With the standard Confidence of 95%, it tells me that the data fails the confidence test. But if I put it up at 99.5% (which says to me that I want to be really confident), then it tells me that if fits the data.
This seems to be backwards. It would seem that if I do not want to be too confident (say 50%), that then it would say that it fits the data. But, if I want to be really confident, I would think that the test would be more restrictive, not less.
Here is the test:
If (1 - confidence) <= significance then it passes, otherwise it fails.
So, with a significance of .005, and the confidence of .95, (1 - .95) > .005
It fails.
But, with a confidence of .995, (1 - .995) = .005
It passes.
| |
Solution:
| When performing a statistical test there is generally one null hypothesis (the data is normally distributed with mean = ? and Sigma = ?) and one or more alternate hypotheses (the data are normal with a different mean and sigma and/or the data are not normally distributed). When you accept the data as normal, you cannot say that you are 95% confident that it is normal. The null is formulated and tested as if it were true.
If you reject the null, you gave it every chance for acceptance (assuming it was true for making all the test calculations, for example) and you would be confident (95%) that it was not normal (again with that mean & sigma). What you can say at this point is that you are 95% confident that the data is not normal with that mean and sigma. That is one of the reasons you say "accept the null" and don't say "prove the null."
The statement you are effectively making is that the data does not disagree with the null. The problem is that there are hundreds of other hypotheses that the data does not disagree with as well. When doing research for a new drug, we would formulate the null as "the new drug is not effective" (no better than a placebo) with some confidence level (95%). If we reject the null, we would be 95% confident that the new drug is effective. If we want to be even more sure that the new drug is effective, we would increase the confidence level to 99% and if we still reject the null, we would be 99% sure (confident) that the new drug is effective.
Gordon Constable, Ph.D. | |
