Activity: Sampling Distributions

Monte Carlo Methods Activities

Author

Manuel D. Rossetti, PhD P.E.

We write this program together: the instructor types the code, and you type along. Before each run, write down a prediction; then run and check it.

The question. Let \(X_1, X_2, \ldots, X_n\) represent a random sample of size \(n\) from a population with mean \(\mu = E[X]\) and variance \(\sigma^2 = \mathrm{Var}[X]\). Let \(\bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i\) represent the sample average. The sample average is itself a random variable: a new sample gives a new average. The central limit theorem says that, as \(n\) grows, the distribution of \(\bar{X}\) approaches a normal distribution with mean \(\mu\) and variance \(\sigma^2/n\). We check this by experiment: draw many samples, average each one, and look at the averages.

Setup. Get the starter file SamplingDistributions.kt from the code/ folder beside this activity. Put it in src/main/kotlin/student/activities in your KSL project, the package the file declares; create the folder if you do not have it. Run its main. It compiles and runs, but the statistics are NaN: the two functions marked TODO are what we write.

1. What to expect

The program draws 50 samples, each of size \(n = 10\), from a normal population with mean 5 and variance 4, written \(N(5, 4)\).

  1. The KSL’s NormalRV(mean, variance) takes the variance as its second parameter. What is the standard deviation of the population? What would go wrong if we wrote NormalRV(5.0, 2.0) thinking of the standard deviation?

  2. Predict the mean, the variance, and the standard deviation of the 50 averages.

  3. Of the 50 averages, about how many do you expect to fall more than two standard deviations of \(\bar{X}\) from 5?

2. A normal population

We write sampleAverages (TODO 1) together.

  1. Run the program. Compare the mean, variance, and standard deviation of the 50 averages with your predictions in part 1. Are they equal? Should they be?

  2. The program also shows the averages against the \(N(5, 0.4)\) distribution in four plots. Do the averages look like a sample from \(N(5, 0.4)\)?

  3. Predict the standard deviation of the averages for samples of size \(n = 40\). Change sampleSize to 40, run again, and check.

3. A uniform population

The program then draws 50 samples from a uniform population on \([10, 60]\), written \(U(10, 60)\). Set sampleSize back to 10.

  1. Predict the mean and the variance of the averages. Recall that \(\mathrm{Var}[X] = (b - a)^2/12\) for \(X \sim U(a, b)\).

  2. The population is flat, not bell shaped. Do you expect the averages of samples of size 10 to look normal? Run and check against the plots.

  3. Both runs estimate the variance of \(\bar{X}\) from only 50 averages. How close should that estimate be to the theory?

4. Confidence interval coverage

Each sample of size 10 gives a 95% confidence interval for the mean. We write coverage (TODO 2) together: it draws 1000 samples from \(N(5, 4)\), and counts the intervals that contain the true mean, 5.

  1. Predict the fraction of the 1000 intervals that contain 5, and the half-width of the estimate of that fraction.

  2. Run. Is the result consistent with a 95% confidence interval? What does “95% confidence” mean, in terms of this experiment?

  3. Each interval uses the Student \(t\) distribution, which assumes the population is normal. Why does the coverage of 0.95 not depend on the sample size here?

If you finish early

Add a fourth population: an exponential distribution with mean 5 (stream 4). Predict, then estimate, the coverage of the 95% intervals from samples of size 10. Why does it differ from part 4?