SAMPLING & DESCRIPTIVE STATS

Chapter 7 of your statistics book, Sampling and Descriptive Statistics , introduces the fundamental transition from theoretical probability (what should happen) to practical statistics (what we observe and infer). It focuses on using data collected from a sample to estimate and describe unknown properties of the larger population distribution.

The core idea is that when a distribution is unknown, we can look at the empirical distribution derived solely from the sampled data, and use it to estimate key characteristics like the average and the variance.

7.1 The Statistical Context

Concept: Probability vs. Statistics

The book draws a distinction: Probability involves studying experiments when the model (the underlying distribution or laws) is fully known. Statistics involves trying to infer unknown aspects of that model based on observed outcomes.

For instance, if we flip a coin 100 times, the results ($X_1, X_2, \dots, X_{100}$) are random variables derived from a Bernoulli distribution with an unknown probability $p$. Statistics seeks to use the 60 heads observed in the sample ($\sum X_i = 60$) to say something about the unknown, long-run probability $p$.

Concept: The Sample

We typically model repeated measurements (like height, weight, or arsenic content) taken from a population as independent and identically distributed (i.i.d.) random variables ($X_1, X_2, \dots, X_n$). While sampling is sometimes done “without replacement,” if the sample size ($n$) is small compared to the population size, the results are highly similar to i.i.d. sampling “with replacement”.

7.1 The Empirical Distribution

Since the true underlying distribution of the population is unknown, the empirical distribution is defined purely from the observed data points.

Concept and Definition

Intuition: If you have five data points $\{1, 1, 2, 4, 6\}$, the empirical distribution states that $P(X=1) = 2/5$, and $P(X=t) = 1/5$ for $t \in \{2, 4, 6\}$.

Key Properties

1. Randomness: The empirical distribution is itself a random quantity, as every new set of sampled data will generate a different distribution.
2. Inference: It is expected that as the sample size ($n$) grows, the empirical distribution should converge to and provide information about the true, underlying distribution.

7.2 Descriptive Statistics

Descriptive statistics are summary numerical measures derived from the empirical distribution to estimate the actual parameters of the underlying distribution.

7.2.1 Sample Mean ($\bar{X}$)

The sample mean is the long-familiar average of the observed data points. It serves as an estimator for the true population mean ($\mu$).

• Unbiased Estimator: $E[\bar{X}] = \mu$. On average, $\bar{X}$ accurately targets the unknown population mean.
• Consistency: $SD[\bar{X}] = \frac{\sigma}{\sqrt{n}}$. As $n \to \infty$, the sample mean converges to the true mean.

7.2.2 Sample Variance ($S^2$)

The sample variance estimates the true population variance ($\sigma^2$).

• Note: Using $(n-1)$ ensures that $S^2$ is an unbiased estimator ($E[S^2] = \sigma^2$).

7.2.3 Sample Proportion ($\hat{p}$) and Empirical CDF

When an event $A$ occurs in the population with probability $p = P(X \in A)$, the sample version of this is the sample proportion ($\hat{p}$).

The sample proportion is an excellent estimator: $E[\hat{p}] = p$ and $Var(\hat{p}) = \frac{p(1-p)}{n}$.

The sample analog of the CDF is the Empirical Cumulative Distribution Function (ECDF): $F_n(t) = \frac{\# \{X_i \leq t\}}{n}$

Example: Sample Mean and Variance