Summarising Continuous Random Variables

SUMMARISING CONTINUOUS RVs

Chapter 6 of your statistics book, Summarising Continuous Random Variables , is the natural continuation of Chapter 4, translating the concepts of average (Expected Value) and spread (Variance/Standard Deviation) from the discrete world (sums) to the continuous world (integrals). It also introduces Moment Generating Functions as powerful tools for working with distributions and concludes by defining Bivariate Normal distributions.

6.1 Expected Value ( $E[X]$ ): The Continuous Average

The expected value, or mean ( $\mu$ ), of a continuous random variable $X$ is the measure of the “long-run average” or the theoretical center of its probability density function (PDF), $f(x)$ .

Concept and Calculation

💡

Continuous Expectation

In the continuous setting, the summation used for discrete RVs is replaced by integration over the entire range of possible outcomes:

$E[X] = \int_{-\infty}^{\infty} x f(x) dx$

If this integral converges to a real number, the variable has finite expectation.

Example 1: Expected Value of a Uniform Distribution

Uniform Expectation

Concept: The uniform distribution is the simplest continuous distribution, where the density is constant over a given interval. Intuitively, the average should be the midpoint of that interval.

Question: What is the expected value of $X \sim \text{Uniform}(a, b)$ ?

View Detailed Solution ▼

Solution: The density is $f(x) = 1/(b-a)$ for $a < x < b$ . $E[X] = \int_{a}^{b} x \cdot \frac{1}{b-a} dx = \frac{1}{b-a} \left[ \frac{x^2}{2} \right]_{a}^{b} = \frac{1}{2(b-a)} (b^2 - a^2) = \frac{b+a}{2}$

Key Properties of Expected Value

The linearity properties established for discrete variables hold true for continuous variables as well, using integrals instead of sums:

$E[aX] = aE[X]$
$E[X + Y] = E[X] + E[Y]$
$E[aX + bY] = aE[X] + bE[Y]$

6.1 Variance and Standard Deviation: The Continuous Spread

Variance and standard deviation quantify the spread of a continuous random variable around its expected value.

Concept and Calculation

💡

Continuous Variance

The variance ( $\text{Var}[X]$ ) is the expected value of the squared distance between $X$ and its mean, $\mu = E[X]$ :

$Var[X] = E[(X - E[X])^2] = \int_{-\infty}^{\infty} (x - E[X])^2 f_X(x) dx$

The standard deviation ( $\text{SD}[X]$ or $\sigma$ ) is the square root of the variance.

Key Properties of Variance

The properties mirror the discrete case:

Alternate Formula: $Var[X] = E[X^2] - (E[X])^2$ .
Scaling: $Var[aX] = a^2 \cdot Var[X]$ .
Shifting: $Var[X + a] = Var[X]$ .
Independence (Product): If $X$ and $Y$ are independent, $E[XY] = E[X]E[Y]$ .
Independence (Sum): If $X$ and $Y$ are independent, $Var[X + Y] = Var[X] + Var[Y]$ .

Example 2: Variance of a Uniform Distribution

Uniform Variance

Question: What is the variance of $X \sim \text{Uniform}(a, b)$ ? (We know $E[X] = (a+b)/2$ ).

View Detailed Solution ▼

Solution: Using the formula $\text{Var}[X] = E[X^2] - (E[X])^2$ :

First, calculate $E[X^2]$ : $E[X^2] = \int_{a}^{b} x^2 \cdot \frac{1}{b-a} dx = \frac{1}{3(b-a)} (b^3 - a^3) = \frac{b^2 + ab + a^2}{3}$

Next, subtract the squared mean: $Var[X] = \frac{b^2 + ab + a^2}{3} - \left( \frac{b+a}{2} \right)^2 = \frac{4(b^2+ab+a^2) - 3(b^2+2ab+a^2)}{12}$ $Var[X] = \frac{b^2 - 2ab + a^2}{12} = \frac{(b-a)^2}{12}$

The standard deviation is $SD[X] = \frac{b-a}{\sqrt{12}}$ .

Application: Central Theorems

The well-known inequalities from discrete probability generalize to the continuous domain. These bounds apply universally, regardless of the specific shape of the continuous distribution.

Markov’s Inequality

For a non-negative continuous RV

X

with finite mean

\mu

, and any

c>0

P(X \geq c) \leq \frac{\mu}{c}

Chebychev’s Inequality

For any continuous RV

X

with finite non-zero variance

\sigma^2

, and any

k>0

P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}

6.2 Conditional Expectation and Variance

For continuous random variables $X$ and $Y$ with a joint density $f(x, y)$ , knowing the outcome of $X$ changes the expected value and spread of $Y$ . This leads to the concepts of conditional expectation and conditional variance.

Concept: Conditional Expectation

The conditional expectation of $Y$ given $X=x$ uses the conditional density $f_{Y|X=x}(y)$ :

$E[Y | X=x] = \int_{-\infty}^{\infty} y f_{Y|X=x}(y) dy = \int_{-\infty}^{\infty} y \frac{f(x, y)}{f_X(x)} dy$

Where $f_X(x)$ is the marginal density of $X$ .

Laws of Total Expectation and Variance

These powerful theorems allow the overall (unconditional) average and variance of a variable to be computed from its conditional characteristics.

Theorem	Formula	Description
Law of Total Expectation	$E[X] = E[E[X\\|Y]]$	Expectation of the conditional expectation.
Law of Total Variance	$Var[X] = E[Var[X\\|Y]] + Var[E[X\\|Y]]$	Sum of expected conditional variance and variance of conditional expectation.

6.2 Covariance and Correlation: Measuring Relationships

When dealing with two continuous random variables, covariance and correlation measure the strength and direction of their linear relationship.

Concept: Covariance

💡

Covariance Formula

The covariance ( $\text{Cov}[X, Y]$ ) measures the degree to which $X$ and $Y$ move together.

$Cov[X,Y] = E[(X - E[X])(Y - E[Y])] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x-\mu_X)(y-\mu_Y) f(x, y) dx dy$

The alternate computational formula is: $Cov[X,Y] = E[XY] - E[X]E[Y]$

A key consequence: If $X$ and $Y$ are independent, then $\mathbf{Cov[X,Y] = 0}$ (they are uncorrelated). (Note: The converse is generally not true).

Concept: Correlation

The correlation coefficient ( $\rho[X, Y]$ ) is the standardized version of covariance, restricted between $-1$ and $1$ :

$\rho[X,Y] = \frac{Cov[X,Y]}{\sigma_X \sigma_Y}$

It is dimensionless and measures the degree of linear association.

6.3 Moment Generating Functions (MGFs)

The moment generating function ( $M(t)$ ) is a mathematical tool that, when it exists, can fully define the distribution of a random variable and simplify complex calculations.

Definition and Moments

The $k$ -th moment of $X$ is $m_k = E[X^k]$ . The MGF is defined as the expected value of $e^{tX}$ :

$M(t) = E[e^{tX}]$

The MGF generates moments via its derivatives evaluated at $t=0$ : $E[X^k] = M^{(k)}(0)$ .

Key Properties of MGFs

Linear Transformation: $M_{aX}(t) = M_X(at)$ .
Sum of Independents: If $X$ and $Y$ are independent, the MGF of their sum is the product of their individual MGFs: $M_{X+Y}(t) = M_X(t) M_Y(t)$
Uniqueness Theorem: If two random variables have the same MGF over an open interval containing 0, they have the exact same distribution.

Example 3: Sum of Independent Normals (MGF Application)

MGF of Normal Sum

Question: If $X \sim \text{Normal}(\mu_1, \sigma_1^2)$ and $Y \sim \text{Normal}(\mu_2, \sigma_2^2)$ are independent, what is the distribution of their sum?

View Detailed Solution ▼

Solution: $M_{X+Y}(t) = M_X(t) M_Y(t)$ Since the MGF for a single normal variable is $M(t) = e^{\mu t + (1/2)\sigma^2 t^2}$ , the resulting MGF of the sum is: $M_{X+Y}(t) = e^{\mu_1 t + (1/2)\sigma_1^2 t^2} \cdot e^{\mu_2 t + (1/2)\sigma_2^2 t^2} = e^{(\mu_1 + \mu_2) t + (1/2)(\sigma_1^2 + \sigma_2^2) t^2}$

By the uniqueness theorem, $X+Y$ must be distributed as $\mathbf{\text{Normal}(\mu_1 + \mu_2, \sigma_1^2 + \sigma_2^2)}$ .

6.4 Bivariate Normal Distributions

This section discusses the properties of a joint distribution where linear combinations of the constituent variables are always normally distributed.

Definition and Properties

💡

Bivariate Normal

A pair of random variables $(X, Y)$ is bivariate normal if every linear combination $aX + bY$ is a normally distributed random variable.

Marginal Normality: $X$ and $Y$ individually are also normally distributed.
Determination by Moments: The joint distribution is completely determined by the means, variances, and correlation.
Independence is Uncorrelation: For bivariate normal variables, $Cov[X, Y] = 0$ if and only if $X$ and $Y$ are independent.

Analogy: Summarizing a continuous random variable is like weighing an object. The Expected Value is the measurement of the object’s mass (its central tendency), while the Variance is a measure of the precision of the scale (how spread out the possible readings are). Covariance, then, is like simultaneously weighing two interconnected objects to see how much one influences the other’s reading.

All Chapters in this Book

Lesson 1

Basic Concepts

Foundational mathematical framework for probability, including definitions, axioms, conditional probability, and Bayes' Theorem.

Lesson 2

Sampling and Repeated Trials

Models based on repeated independent trials, focusing on Bernoulli trials and sampling methods.

Lesson 3

Discrete Random Variables

Formalizing random variables, probability mass functions, and independence.

Lesson 4

Summarizing Discrete Random Variables

Deriving numerical characteristics—expected value, variance, and standard deviation—to summarize behavior of discrete random variables.

Lesson 5

Continuous Probabilities and Random Variables

Transitioning from discrete sums to continuous integrals, density functions, and key distributions like Normal and Exponential.

Lesson 6