|
|
|
 |
|
|
|
A First Look at Significance and the p-value Through Simulation
|
|
| |
by Jim Bohan
Program Coordinator
Manheim Township School District
Lancaster, Pennsylvania
|
|
| |
The concepts of significance and the p-value are critical to understanding statistical inference in the form of tests of significance. One of the major concerns of teachers of statistics should be whether students understand the concept of significance and the meaning of the p-value for a test of significance. This concern is exacerbated by the easy access to calculator and computer techniques that will produce a p-value in a completely procedural way.
I have used the following progression of ideas with my AP Statistics students and have found that they develop a good conceptual basis for tests of significance and the p-value before we discuss the Central Limit Theorem.
1. Understanding the concept of a sampling distribution.
I believe it is critical that students create and explore many sampling distributions with different population distributions and different statistics of interest so that they eventually integrate the concept of a sampling distribution into their understanding. I use Fathom (Key Curriculum Press) for this purpose and have developed several Fathom files under the title heading of CLT for the Central Limit Theorem.
For example, the figure below depicts the sampling distribution of sample means from a highly skewed population from which 50 cases are selected. The figure displays and analyzes the distribution of the means of 100 random samples (with replacement).
This figure depicts the sampling distribution of sample medians from a highly skewed population from which again 50 cases are selected. The figure displays and analyzes the sampling distribution from 100 random samples.
In all cases, students are asked to describe the elements, shape, and measures of the sampling distributions that are found. At this stage of the development of the concepts, we hopefully plant the seed in their minds that some sampling distributions can be identified using one of the probability distributions that we have studied earlier, while others cannot.
2. Definition of the p-value.
Authors differ in the precise definition of the p-value: Some define this number as the probability that a random sample produces a test statistic of as extreme or more extreme when the parameter in the null hypothesis is true. Some suggest that the p-value should only be discussed after the sampling distribution has been identified. Other authors define the p-value as the probability that a sample produces a statistic this extreme or more extreme if the parameter in the null hypothesis is true.
For example, using this latter definition, the p-value for a test of H0:µ = 40 versus Ha: µ > 40, based on a sample of size 50 with a sample mean of 42.5 and sample standard deviation of 10.55, can be expressed as:
I prefer this definition as it allows us to discuss the meaning of the p-value before we discuss the issues of the Central Limit Theorems and the identification of sampling distributions. We can estimate the value of the p-value using an enhancement to the sampling distribution simulations that we developed in our initial discussion of sampling distributions.
3. Estimating a p-value via simulations.
I wish to emphasize that this estimation of the p-value is completed before there is any discussion of a Central Limit Theorem. I believe that the students should experience p-values in advance of a formal investigation of particular sampling distributions.
This figure displays the outcome of a simulation for the test of significance detailed above. In this simulation, we have determined that an estimated p-value is 0.04. Needless to say, if the number of samples were to increase, the estimate would improve.
After the Central Limit Theorem for sample means has been developed for the students, we return to this test and calculate that the p-value for this test using a t-test is 0.050.
The figure below displays the outcome of a simulation determining the p-value of a test of significance for a median -- H0: population median = 3.35 versus Ha: population median > 3.35, with a sample median = 3.90 from a sample of 50 cases and a sampling distribution including medians from 100 samples. For this test, the p-value has the meaning:
Pr (sample of size 50 produces a sample median > 3.90 when the true median is 3.35)
From this simulation, the value of the p-value is 0.13. Calculation of the p-value for a test of medians is not included in the syllabus for AP Statistics. However, clearly, we can have the students experience a test for medians or any other measure using simulation.
The figure below displays the outcome of a simulation determining the p-value of a test of significance for a standard deviation: H0: 
= 2.867 versus Ha:
> 2.867, with an s = 3.30 from a sample of 50 cases and a sampling distribution including medians from 100 samples. For this test, the p-value has the meaning:
Pr (sample of size 50 produces a sample standard deviation > 3.30
when the true standard deviation is 2.867)
In conclusion, I suggest that using simulation and a "broader" definition of the p-value, we can have students experience and explore the concept of significance in advance of a formal discussion of the Central Limit Theorem. With this approach, the Central Limit Theorem becomes a tool for identifying sampling distributions and for calculating the p-value.
Jim Bohan is the K-12 mathematics program coordinator at Manheim Township School District in Lancaster, Pennsylvania. He is a veteran of 35 years teaching in parochial, private, and public high schools, and has been an adjunct instructor of mathematics and statistics at several colleges and universities in Illinois and his present home of Pennsylvania. Jim is a consultant to the College Board for AP Calculus and AP Statistics, and has been a member of the AP Statistics Development Committee.
|
|
|
|
|
|
Course Home Pages