Blog

ETF2121/ETF5912 Data Analysis in Business
Week 4: Hypothesis testing and prediction for one population
Dr Wei Wei
Monash University
Dr Wei Wei (Monash University) ETF2121/5912 1 / 41
1 Hypothesis testing
Setting up the hypotheses
Specifying a significance level
Estimation, sampling distribution, and test statistic
Making the decision
Hypothesis tests and confidence interval
Summary
2 Prediction
Prediction interval and confidence interval
Dr Wei Wei (Monash University) ETF2121/5912 2 / 41
Hypothesis testing
Design of Hypothesis Testing
There are five steps in the process of hypothesis testing:
1 Formulate the hypotheses: set up the null (H0) and alternative (Ha)
hypotheses for the question at hand.
2 Specify the level of significance, α.
3 Determine the appropriate estimator, test statistic and its sampling
distribution.
4 Compute the value of the test statistic from the sample.
5 Make a decision (of whether or not to reject the null) using one of the
following approaches:
1 the p-value approach;
2 the critical value approach.
Dr Wei Wei (Monash University) ETF2121/5912 3 / 41
Hypothesis testing Setting up the hypotheses
Step 1: setting up the hypotheses
The null hypothesis, H0, is assumed to be true in the absence of
contradictory evidence.
The alternative hypothesis, HA, is designed to answer a specific
question.
In two-sided (two-tailed) tests, H0 states that the population
parameter is equal to a single value, while HA states that the
population parameter is not equal to that value.
H0 : θ = θ0
HA : θ 6= θ0
In one-sided (one-tailed) tests, H0 states that the population
parameter is equal to a single value, while HA states that the
population parameter is greater than or less than that value.
case 1:
H0 : θ = θ0
HA : θ > θ0
case 2:
H0 : θ = θ0
HA : θ < θ0 Dr Wei Wei (Monash University) ETF2121/5912 4 / 41 Hypothesis testing Setting up the hypotheses Example 4.1: sugar content in breakfast cereal The manufacturer of Cocoa Puffs claims that average sugar content in 100g of this cereal is 25.7g. A dietitian want to check if the average sugar content in Cocoa Puffs is indeed what the company claims. She obtains a sample of 50 boxes, each with 100g of cocoa puffs, and measures the sugar content in the sample. Let μ denote the population parameter, state the null and the alternative hypotheses. Dr Wei Wei (Monash University) ETF2121/5912 5 / 41 Hypothesis testing Setting up the hypotheses Example 4.2: pizza waiting times A pizza outlet advertises that its average waiting time is 12 minutes from the time an order is placed. A frequent customer claims that the average waiting time is more than the advertised time and presented waiting times for his last 10 orders as evidence. Let μ denote the population parameter, state the null and the alternative hypotheses. Dr Wei Wei (Monash University) ETF2121/5912 6 / 41 Hypothesis testing Setting up the hypotheses Example 4.3: are voting results manipulated In a recent election, candidate A is reported to received 45% of the vote. However, candidate A believes that the voting results are manipulated and more people must have voted for him. Candidate A hires a consulting agency to randomly sample 1000 voters to record whether or not each person voted for him. Let pi denote the population parameter, state the null and the alternative hypotheses. Dr Wei Wei (Monash University) ETF2121/5912 7 / 41 Hypothesis testing Setting up the hypotheses Example 4.4: standard deviation of shafts Company ABC manufactures steel shafts with a standard length of 20cm. The industry standard allows a standard deviation of 2cm in the production process. Company ABC claims that they have better quality control and the standard deviation of the lengths of their shafts is smaller than the industry standard. They provided length of 50 shafts from a recently produced batch as evidence. Let σ denote the population parameter, state the null and the alternative hypothesis. Dr Wei Wei (Monash University) ETF2121/5912 8 / 41 Hypothesis testing Setting up the hypotheses Example 4.5: presumption of innocence In most legal systems around the world, a person accused of any crime is “presumed innocent until proven guilty”. How does the presumption of innocence translate to the null and alternative hypothesis Dr Wei Wei (Monash University) ETF2121/5912 9 / 41 Hypothesis testing Specifying a significance level Type I and type II errors In hypothesis testing, we can make two kinds of mistakes. Type I error: the probability of rejecting H0 given that H0 is true. Type II error: the probability of not rejecting H0 given that H0 is false. State of the world H0 is true HA is true Reject Type I error: Pr(R|H0) Correct Do not Reject Correct Type 2 error: Pr(NR|HA) Type I and type II errors are inversely related; generally speaking, we can not decrease one error without increasing the other unless we change the sample/estimator. Dr Wei Wei (Monash University) ETF2121/5912 10 / 41 Hypothesis testing Specifying a significance level Step 2: specifying a significance level Hypothesis tests are constructed to control type I error. Specifically, we define the significance level = Pr(R|H0) and denote it by α. In economics/business, 1%, 5% or 10% are the commonly used significance level. In other words, α = 0.01, 0.05 or 0.1. Which α to use depends on how we evaluate the cost of each error. α evidence needed to reject the null probability of type 2 error Preferred if the loss from type one error is 0.01 stronger higher higher 0.05 0.10 weaker lower lower Dr Wei Wei (Monash University) ETF2121/5912 11 / 41 Hypothesis testing Specifying a significance level Example 4.2: pizza waiting times A pizza outlet advertises that its average waiting time is 12 minutes from the time an order is placed. A frequent customer claims that the average waiting time is more than its advertised time and presented waiting times for his last 10 orders as evidence. The null and alternative hypothesis is given below H0 : μ = 12 HA : μ > 12
Two statisticians, A and B, are asked to evaluate if there’s enough
evidence to support the customer’s claim. Statistician A cares more
about protecting customer rights, while statistician B cares more
about protecting the rights of small business owners.
Interpret type 1 and type 2 error in this example.
Which statistician would use a higher significance level for the test
above
Dr Wei Wei (Monash University) ETF2121/5912 12 / 41
Hypothesis testing Specifying a significance level
Example 4.5: presumption of innocence
In most legal systems around the world, a person accused of any crime
is “presumed innocent until proven guilty”.
The English jurist William Blackstone once wrote “It is better that ten
guilty persons escape than that one innocent suffer.” This has become
a maxim known as the Blackstone ratio. The German politician
Bismarck has “allegedly” said “it is better that ten innocent men suffer
than one guilty man escape”.
Interpret type 1 and type 2 error in this example.
If these two are using statistical evidence to make a decision, who
would use a higher significance level
Dr Wei Wei (Monash University) ETF2121/5912 13 / 41
Hypothesis testing Estimation, sampling distribution, and test statistic
Step 3: estimation, sampling distribution, and test statistic
1 Work out an appropriate sample estimator for the population
parameter.
2 Work out the sampling distribution for the estimator under the null
hypothesis. In practice, we transform/standardize the estimator using
the null to obtain a test statistic with a known distribution.
3 The distribution of a test statistic under the null are also called the
null distribution.
Dr Wei Wei (Monash University) ETF2121/5912 14 / 41
Hypothesis testing Estimation, sampling distribution, and test statistic
Example 4.1: sugar content in breakfast cereal
The manufacturer of Cocoa Puffs claims that average sugar content in
100g of this cereal is 25.7g.
A dietitian want to check if the average sugar content in Cocoa Puffs
is indeed what the company claims. She obtains a sample of 50 boxes,
each with 100g of cocoa puffs, and measures the sugar content in the
sample.
What is an appropriate estimator for the population parameter
What is the sampling distribution of this estimator under the null
(assume that the sugar content follows a normal distribution with
standard deviation equal to 5g)
Standardize the estimator under the null to obtain the test statistic.
What is the null distribution of the test statistic
Dr Wei Wei (Monash University) ETF2121/5912 15 / 41
Hypothesis testing Estimation, sampling distribution, and test statistic
Example 4.2: pizza waiting times
A pizza outlet advertises that its average waiting time is 12 minutes
from the time an order is placed.
A frequent customer claims that the average waiting time is more than
the advertised time and presented waiting time for his last 10 orders as
evidence.
What is an appropriate estimator for the population parameter
Assume that waiting times follow a normal distribution with unknown
standard deviation. Standardize the estimator under the null and state
its distribution.
Dr Wei Wei (Monash University) ETF2121/5912 16 / 41
Hypothesis testing Estimation, sampling distribution, and test statistic
Example 4.3: are voting results manipulated
In a recent election, candidate A is reported to received 45% of the
vote. However, candidate A believes that the voting results are
manipulated and more people must have voted for him.
Candidate A hires a consulting agency to randomly sample 1000 voters
to record whether or not each person voted for him.
What is an appropriate estimator for the population parameter
What is the sampling distribution of this estimator under the null
Standardize the estimator under the null to obtain the test statistic.
What is the null distribution of the test statistic
Dr Wei Wei (Monash University) ETF2121/5912 17 / 41
Hypothesis testing Estimation, sampling distribution, and test statistic
Example 4.4: standard deviation of shafts
Company ABC manufactures steel shafts with a standard length of
20cm. The industry standard allows a standard deviation of 2cm in
the production process. Company ABC claims that they have better
quality control and the standard deviation of the lengths of their shafts
is smaller than the industry standard. They provided length of 50
shafts from a recently produced batch as evidence.
What is an appropriate estimator for the population parameter
Assume that the length of shafts follows a normal distribution.
Standardize the estimator under the null and state its distribution.
Dr Wei Wei (Monash University) ETF2121/5912 18 / 41
Hypothesis testing Making the decision
Step 4 and 5: compute the test statistic and make the
decision
Step 4: Using the estimator in step 3 to compute the sample estimate
and the test statistic.
Step 5: Using one of the two following methods to make the decision
a: the p-value approach
b: the critical value approach
Dr Wei Wei (Monash University) ETF2121/5912 19 / 41
Hypothesis testing Making the decision
Step 5a: Hypothesis testing using p-value
A p-value represents the probability that the sample statistic (test
statistic) takes the observed value or more extreme values if the null
hypothesis were true.
To find the p-value, we use the distribution of the test statistic, and
find the tail area that corresponds to the probability of observing the
computed test statistic or more extreme values.
Decision rule:
reject if p < α do not reject if p > α
Dr Wei Wei (Monash University) ETF2121/5912 20 / 41
Hypothesis testing Making the decision
Step 5a: Computing p-values
Let X denote the test statistic with a known distribution and x denote
the value of the test statistic computed from the sample;
Depending on the alternative hypothesis, p-values can be computed
from
Ha : θ > θ0 Ha : θ < θ0 Ha : θ 6= θ0 p = Pr(X > x) p = Pr(X < x) p = 2Pr(X > |x |)
If the test statistic has a symmetric distribution around zero, then
Pr(X > x) = Pr(X < x) Dr Wei Wei (Monash University) ETF2121/5912 21 / 41 Hypothesis testing Making the decision Step 5b: Hypothesis testing using the critical value approach Alternatively, we can use the critical value approach. Let X denote the test statistic with a known distribution and x denote the value of the test statistic computed from the sample; The critical values are percentiles of X . Let Xα denote the 100αth percentile of X , then depending on the alternative hypothesis Ha : θ > θ0 Ha : θ < θ0 Ha : θ 6= θ0 Critical values X1 α Xα Xα/2 and X1 α/2 Reject if x > X1 α x < Xα x < Xα/2 or x > X1 α/2
Do not reject if x < X1 α x > Xα Xα/2 < x < X1 α/2 Dr Wei Wei (Monash University) ETF2121/5912 22 / 41 Hypothesis testing Making the decision Example 4.1: sugar content in breakfast cereal The manufacturer of Cocoa Puffs claims that average sugar content in 100g of this cereal is 25.7g. A dietitian want to check if the average sugar content in Cocoa Puffs is indeed what the company claims. She obtains a sample of 50 boxes, each with 100g of cocoa puffs, and measures the sugar content in the sample. The mean sugar content from the sample of 50 boxes is 27.4g. Do you reject the null at the 5% significance level (assume that the sugar content follows a normal distribution with standard deviation equal to 5g) Dr Wei Wei (Monash University) ETF2121/5912 23 / 41 Hypothesis testing Making the decision Example 4.1: sugar content in breakfast cereal (p-value approach) Recall that Pr(Z > z) = Pr(Z < z) = Φ( z). In EXCEL Φ( z) = NORM.DIST(-z, 0,1,1) Dr Wei Wei (Monash University) ETF2121/5912 24 / 41 Hypothesis testing Making the decision Example 4.1: sugar content in breakfast cereal (critical value approach) Recall that in EXCEL z0.975 = NORM.INV(0.975,0,1) z0.025 = NORM.INV(0.025,0,1) or z0.025 = z0.975 Dr Wei Wei (Monash University) ETF2121/5912 25 / 41 Hypothesis testing Making the decision Example 4.2: pizza waiting times A pizza outlet advertises that its average waiting time is 12 minutes from the time an order is placed. A frequent customer claims that the average waiting time is more than the advertised time and presented waiting time for his last 10 orders as evidence. The sample mean of the waiting time is 13 minutes and the sample standard deviation is 2 minutes. Do you reject the null at the 5% significance level Dr Wei Wei (Monash University) ETF2121/5912 26 / 41 Hypothesis testing Making the decision Example 4.2: pizza waiting times (p-value) Recall that Pr(T > t) = Pr(T < t). In EXCEL Pr(T < t) = T.DIST(-t, n-1,1) Pr(T > t) =1-T.DIST(t, n-1,1)
Dr Wei Wei (Monash University) ETF2121/5912 27 / 41
Hypothesis testing Making the decision
Example 4.2: pizza waiting times (critical value)
Recall that in EXCEL
t0.95 =T.INV(0.95, n-1)
Dr Wei Wei (Monash University) ETF2121/5912 28 / 41
Hypothesis testing Making the decision
Example 4.3: are voting results manipulated
In a recent election, candidate A is reported to received 45% of the
vote. However, candidate A believes that the voting results are
manipulated and more people must have voted for him.
Candidate A hires a consulting agency to randomly sample 1000 voters
to record whether or not each person voted for him.
510 voter responded that they voted for candidate A. Do you reject
the null at the 5% significance level
Dr Wei Wei (Monash University) ETF2121/5912 29 / 41
Hypothesis testing Making the decision
Example 4.3: are voting results manipulated
Dr Wei Wei (Monash University) ETF2121/5912 30 / 41
Hypothesis testing Making the decision
Example 4.4: standard deviation of shafts
Company ABC manufactures steel shafts with a standard length of
20cm. The industry standard allows a standard deviation of 2cm in
the production process. Company ABC claims that they have better
quality control and the standard deviation of the lengths of their shafts
is smaller than the industry standard. They provided length of 50
shafts from a recently produced batch as evidence.
The sample standard deviation of the length of 50 shafts is 1.8cm. Do
you reject the null at the 5% significance level
Dr Wei Wei (Monash University) ETF2121/5912 31 / 41
Hypothesis testing Making the decision
Example 4.4: standard deviation of shafts (p-value
approach)
Recall Pr(Q < q) = CHISQ.DIST(q,n-1,1) Dr Wei Wei (Monash University) ETF2121/5912 32 / 41 Hypothesis testing Making the decision Example 4.4: standard deviation of shafts (critical value approach) Recall χ20.05,n 1 = CHISQ.INV(0.05,n-1) Dr Wei Wei (Monash University) ETF2121/5912 33 / 41 Hypothesis testing Hypothesis tests and confidence interval Hypothesis tests and confidence interval If we are doing a two-sided test, the confidence interval can also be used for hypothesis testing. If the 100(1 α)% confidence interval does not contain the value in the null, we can reject the null at the 100α% level. Dr Wei Wei (Monash University) ETF2121/5912 34 / 41 Hypothesis testing Summary Hypothesis test about a population mean Given the null H0 : μ = μ0 If the population is normally distributed and the population variance is unknown: t = X μ0 s/ √ n ～ tn 1 If the population is normally distributed and the population variance is known: Z = X μ0 σ/ √ n ～ N(0, 1) If the population distribution is unknown and the population variance is unknown, but the sample size is large: Z = X μ0 s/ √ n ～ N(0, 1) Dr Wei Wei (Monash University) ETF2121/5912 35 / 41 Hypothesis testing Summary Hypothesis test about a population mean If the population is normally distributed and the population variance is unknown: Null H0 : μ = μ0 Alternative HA : μ > μ0 HA : μ < μ0 HA : μ 6= μ0 Test statistic t = X μ0 s/ √ n ～ tn 1 p-values Pr(T > t) Pr(T < t) 2Pr(T > |t|)
EXCEL =t.dist(-t,n-1,1) =t.dist(t,n-1,1) =2*t.dist(abs(t),n-1,1)
Critical
values
t1 α,n 1 tα,n 1 ±tα/2,n 1
EXCEL =t.inv(1-α,n-1) =t.inv(α,n-1) =t.inv(α/2,n-1)
Reject if t > t1 α,n 1 t < tα,n 1 t < tα/2,n 1 or t > t1 α/2,n 1
Do not reject
if
t < t1 α,n 1 t > tα,n 1 tα/2,n 1 < t < t1 α/2,n 1 Dr Wei Wei (Monash University) ETF2121/5912 36 / 41 Hypothesis testing Summary Hypothesis test about a population proportion Given the null H0 : pi = pi0 If the sample size is large Z = p pi0√ pi0(1 pi0) n ～ N(0, 1) Dr Wei Wei (Monash University) ETF2121/5912 37 / 41 Hypothesis testing Summary Hypothesis test about a population variance Given the null H0 : σ 2 = σ20 If the population is normally distributed, Q = (n 1)s2 σ20 ～ χ2n 1 Dr Wei Wei (Monash University) ETF2121/5912 38 / 41 Prediction Prediction Prediction/forecasting is another important application in statistics. We distinguish between a point forecast: the expectation of the population variable; a prediction interval: an interval we expect the population variable to lie with a specified probability. For now, we will only consider the case when the population variable is normally distributed and the sample size is large. Let X denote the population variable, X the sample mean, and s2 the sample variance. A new draw from the population follows the distribution X ～ N(X , s2) the point forecast for X is given by the sample mean X the 100(1 α)% prediction interval for X is given by X ± z1 α/2s Dr Wei Wei (Monash University) ETF2121/5912 39 / 41 Prediction Prediction interval and confidence interval Prediction interval and confidence interval If X is normally distributed and the sample size is large. The confidence interval for the population mean is given by X ± z1 α/2 s√ n The prediction interval for the population variable is given by X ± z1 α/2s Dr Wei Wei (Monash University) ETF2121/5912 40 / 41 Prediction Prediction interval and confidence interval Example 4.6: prediction interval and confidence interval Population variable: weekly earning of construction workers Sample: weekly earning of 900 construction worker Sample mean: X = 1000 Sample variance: s2 = 400 Compute and interpret the 95% confidence interval for the population mean. Compute and interpret the 95% prediction interval for the population variable. Dr Wei Wei (Monash University) ETF2121/5912 41 / 41