The first time I have studied “Hypothesis testing” was when I enrolled “Probability and Statistics” with prof. Martinelli, during my master degree.
In the beginning, I didn’t understand easily the topic, but in the following month, practicing and practicing, I became quite confident.
In this post, I will try to explain the Hypothesis Testing, also because I’ve promised to publish this article to Diego, after a call on skype where he helped me to understand how PyCharm and Jupyter work.
Some days ago, Francesco, a friend of mine, discovered from his University Student Office, that the average age of all the enrolled students was 23 years.
Francesco, always skeptics, replied to the University Student Office: “Doubts*”
How could he verify if the statement was reasonable?
He could with “Hypothesis Testing”.
In this specific case, the Hypothesis is on the average age of enrolled students.
How to verify this Hypothesis?
Brutally Francesco needs to verify if the value assumed is not too much different from another value that he is going to determine as checking value.
Simplified and not accurate explanation: We evaluate the probability that the difference between our Hypothesis Value and the Check Value will be higher of a defined threshold.
We accept the hypothesis if the difference is smaller than our threshold value, we reject the hypothesis if greater.
The threshold value is called “Level of Significance of the test”
In the picture is represented the difference between our threshold value X and the Hypothesis Value µ0, if that difference is inside the 95% of the distribution bell, we accept our Hypothesis with a 5% level of significance, otherwise, we reject it.
In other terms, we are stating that the difference between our hypothesis and our threshold is unlikely to be so high.
Level of significance is a key concept in the “Hypothesis Testing”, is a value that describes the probability to reject a Hypothesis that is true.
If Francesco stated that student average age will not be 23 when is true, he rejected a true hypothesis.
Significance level is often expressed in term of α
For example, a 5% level of significance means that we have the 5% of probability to state that our hypothesis is false, and reject it when instead is true.
Francesco doesn’t know if the average age is 23 as University Student Office Stated, it could be 24, 25 or 22.
Each value will be characterized by a probability to be accetable, so it will be likely that the average age will be 23, unlikely 35 or 18.
Francesco takes a random sample of 40 enrolled students and determines the average age of the sampled group.
I will not describe all the mathematic formulas behind the scenes (but I am going to describe in the appendix of the post).
Francesco wants to verify his hypothesis with a 5% level of significance.
If the disequation is true he accepts the hypothesis, otherwise, he rejects it.
There are some points that I have implied and need to be discussed further.
If you had patience you can find above here.
In the following days, I will talk on p-values and A/B testing with Python.
Thanks for reading and if you find any mistake let me know about it, I will fix it, especially grammar errors.
In the example I didn’t say:
µ0= 23 is the mean of the enrolled student age distribution.
Distribution Variance is known.
Sample mean is a natural point estimator of the enrolled student mean distribution, that is unknown
If we state that our Hypothesis is true, follows that the enrolled student mean distribution has a normal distribution
If we accept a hypothesis, for example, that mean value of a distribution is µ0, with a level of significance α (in our example 5%) we are stating that exist a region in probability space c that:
If the sample mean X follows a normal distribution with mean µ0 we can say that Z is a random variable that:
What we are doing, based on a decided level of significance, is to identify the value of the normal standard variable associated with the probability of our threshold value.
97.5% is the probability that our Z assumes a value less than 1.96, or vice-versa, 2.5% is the probability that our Z will be greater than 1.96
What we are saying is ” If the sample mean less the hypothesis mean divided by standard deviation multiplied by the square root of the number of samples is greater than 1.96 with a significance level of 5% follows that the hypothesis is false”
The last point is on the two type of errors that you can make in the hypothesis testing:
- First type, when data conduct us to reject an hypothesis that is true.
- Second type, when data conduct us to accept an hypothesis that is false.
*Dubts is an italian short way to express skepticism to something