Menu

Summarising Continuous Random Variables

SUMMARISING CONTINUOUS RVs

Chapter 6 of your statistics book, Summarising Continuous Random Variables , is the natural continuation of Chapter 4, translating the concepts of average (Expected Value) and spread (Variance/Standard Deviation) from the discrete world (sums) to the continuous world (integrals). It also introduces Moment Generating Functions as powerful tools for working with distributions and concludes by defining Bivariate Normal distributions.

6.1 Expected Value (E[X]E[X]): The Continuous Average

The expected value, or mean (μ\mu), of a continuous random variable XX is the measure of the “long-run average” or the theoretical center of its probability density function (PDF), f(x)f(x).

Concept and Calculation

💡

Continuous Expectation

In the continuous setting, the summation used for discrete RVs is replaced by integration over the entire range of possible outcomes:

E[X]=∫−∞∞xf(x)dxE[X] = \int_{-\infty}^{\infty} x f(x) dx

If this integral converges to a real number, the variable has finite expectation.

Example 1: Expected Value of a Uniform Distribution

Q1

Uniform Expectation

Concept: The uniform distribution is the simplest continuous distribution, where the density is constant over a given interval. Intuitively, the average should be the midpoint of that interval.

Question: What is the expected value of X∼Uniform(a,b)X \sim \text{Uniform}(a, b)?

📝 View Detailed Solution ▼

Solution: The density is f(x)=1/(b−a)f(x) = 1/(b-a) for a<x<ba < x < b. E[X]=∫abx⋅1b−adx=1b−a[x22]ab=12(b−a)(b2−a2)=b+a2E[X] = \int_{a}^{b} x \cdot \frac{1}{b-a} dx = \frac{1}{b-a} \left[ \frac{x^2}{2} \right]_{a}^{b} = \frac{1}{2(b-a)} (b^2 - a^2) = \frac{b+a}{2}

Key Properties of Expected Value

The linearity properties established for discrete variables hold true for continuous variables as well, using integrals instead of sums:

  1. E[aX]=aE[X]E[aX] = aE[X]
  2. E[X+Y]=E[X]+E[Y]E[X + Y] = E[X] + E[Y]
  3. E[aX+bY]=aE[X]+bE[Y]E[aX + bY] = aE[X] + bE[Y]

6.1 Variance and Standard Deviation: The Continuous Spread

Variance and standard deviation quantify the spread of a continuous random variable around its expected value.

Concept and Calculation

💡

Continuous Variance

The variance (Var[X]\text{Var}[X]) is the expected value of the squared distance between XX and its mean, μ=E[X]\mu = E[X]:

Var[X]=E[(X−E[X])2]=∫−∞∞(x−E[X])2fX(x)dxVar[X] = E[(X - E[X])^2] = \int_{-\infty}^{\infty} (x - E[X])^2 f_X(x) dx

The standard deviation (SD[X]\text{SD}[X] or σ\sigma) is the square root of the variance.

Key Properties of Variance

The properties mirror the discrete case:

  1. Alternate Formula: Var[X]=E[X2]−(E[X])2Var[X] = E[X^2] - (E[X])^2.
  2. Scaling: Var[aX]=a2⋅Var[X]Var[aX] = a^2 \cdot Var[X].
  3. Shifting: Var[X+a]=Var[X]Var[X + a] = Var[X].
  4. Independence (Product): If XX and YY are independent, E[XY]=E[X]E[Y]E[XY] = E[X]E[Y].
  5. Independence (Sum): If XX and YY are independent, Var[X+Y]=Var[X]+Var[Y]Var[X + Y] = Var[X] + Var[Y].

Example 2: Variance of a Uniform Distribution

Q2

Uniform Variance

Question: What is the variance of X∼Uniform(a,b)X \sim \text{Uniform}(a, b)? (We know E[X]=(a+b)/2E[X] = (a+b)/2).

📝 View Detailed Solution ▼

Solution: Using the formula Var[X]=E[X2]−(E[X])2\text{Var}[X] = E[X^2] - (E[X])^2:

First, calculate E[X2]E[X^2]: E[X2]=∫abx2⋅1b−adx=13(b−a)(b3−a3)=b2+ab+a23E[X^2] = \int_{a}^{b} x^2 \cdot \frac{1}{b-a} dx = \frac{1}{3(b-a)} (b^3 - a^3) = \frac{b^2 + ab + a^2}{3}

Next, subtract the squared mean: Var[X]=b2+ab+a23−(b+a2)2=4(b2+ab+a2)−3(b2+2ab+a2)12Var[X] = \frac{b^2 + ab + a^2}{3} - \left( \frac{b+a}{2} \right)^2 = \frac{4(b^2+ab+a^2) - 3(b^2+2ab+a^2)}{12} Var[X]=b2−2ab+a212=(b−a)212Var[X] = \frac{b^2 - 2ab + a^2}{12} = \frac{(b-a)^2}{12}

The standard deviation is SD[X]=b−a12SD[X] = \frac{b-a}{\sqrt{12}}.

Application: Central Theorems

The well-known inequalities from discrete probability generalize to the continuous domain. These bounds apply universally, regardless of the specific shape of the continuous distribution.

Markov’s Inequality

For a non-negative continuous RV XX with finite mean μ\mu, and any c>0c>0: P(X≥c)≤μcP(X \geq c) \leq \frac{\mu}{c}

Chebychev’s Inequality

For any continuous RV XX with finite non-zero variance σ2\sigma^2, and any k>0k>0: P(∣X−μ∣≥kσ)≤1k2P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}

6.2 Conditional Expectation and Variance

For continuous random variables XX and YY with a joint density f(x,y)f(x, y), knowing the outcome of XX changes the expected value and spread of YY. This leads to the concepts of conditional expectation and conditional variance.

Concept: Conditional Expectation

The conditional expectation of YY given X=xX=x uses the conditional density fY∣X=x(y)f_{Y|X=x}(y):

E[Y∣X=x]=∫−∞∞yfY∣X=x(y)dy=∫−∞∞yf(x,y)fX(x)dyE[Y | X=x] = \int_{-\infty}^{\infty} y f_{Y|X=x}(y) dy = \int_{-\infty}^{\infty} y \frac{f(x, y)}{f_X(x)} dy

Where fX(x)f_X(x) is the marginal density of XX.

Laws of Total Expectation and Variance

These powerful theorems allow the overall (unconditional) average and variance of a variable to be computed from its conditional characteristics.

TheoremFormulaDescription
Law of Total ExpectationE[X]=E[E[X∥Y]]E[X] = E[E[X\|Y]]Expectation of the conditional expectation.
Law of Total VarianceVar[X]=E[Var[X∥Y]]+Var[E[X∥Y]]Var[X] = E[Var[X\|Y]] + Var[E[X\|Y]]Sum of expected conditional variance and variance of conditional expectation.

6.2 Covariance and Correlation: Measuring Relationships

When dealing with two continuous random variables, covariance and correlation measure the strength and direction of their linear relationship.

Concept: Covariance

💡

Covariance Formula

The covariance (Cov[X,Y]\text{Cov}[X, Y]) measures the degree to which XX and YY move together.

Cov[X,Y]=E[(X−E[X])(Y−E[Y])]=∫−∞∞∫−∞∞(x−μX)(y−μY)f(x,y)dxdyCov[X,Y] = E[(X - E[X])(Y - E[Y])] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} (x-\mu_X)(y-\mu_Y) f(x, y) dx dy

The alternate computational formula is: Cov[X,Y]=E[XY]−E[X]E[Y]Cov[X,Y] = E[XY] - E[X]E[Y]

A key consequence: If XX and YY are independent, then Cov[X,Y]=0\mathbf{Cov[X,Y] = 0} (they are uncorrelated). (Note: The converse is generally not true).

Concept: Correlation

The correlation coefficient (ρ[X,Y]\rho[X, Y]) is the standardized version of covariance, restricted between −1-1 and 11:

ρ[X,Y]=Cov[X,Y]σXσY\rho[X,Y] = \frac{Cov[X,Y]}{\sigma_X \sigma_Y}

It is dimensionless and measures the degree of linear association.

6.3 Moment Generating Functions (MGFs)

The moment generating function (M(t)M(t)) is a mathematical tool that, when it exists, can fully define the distribution of a random variable and simplify complex calculations.

Definition and Moments

The kk-th moment of XX is mk=E[Xk]m_k = E[X^k]. The MGF is defined as the expected value of etXe^{tX}:

M(t)=E[etX]M(t) = E[e^{tX}]

The MGF generates moments via its derivatives evaluated at t=0t=0: E[Xk]=M(k)(0)E[X^k] = M^{(k)}(0).

Key Properties of MGFs

  1. Linear Transformation: MaX(t)=MX(at)M_{aX}(t) = M_X(at).
  2. Sum of Independents: If XX and YY are independent, the MGF of their sum is the product of their individual MGFs: MX+Y(t)=MX(t)MY(t)M_{X+Y}(t) = M_X(t) M_Y(t)
  3. Uniqueness Theorem: If two random variables have the same MGF over an open interval containing 0, they have the exact same distribution.

Example 3: Sum of Independent Normals (MGF Application)

Q3

MGF of Normal Sum

Question: If X∼Normal(μ1,σ12)X \sim \text{Normal}(\mu_1, \sigma_1^2) and Y∼Normal(μ2,σ22)Y \sim \text{Normal}(\mu_2, \sigma_2^2) are independent, what is the distribution of their sum?

📝 View Detailed Solution ▼

Solution: MX+Y(t)=MX(t)MY(t)M_{X+Y}(t) = M_X(t) M_Y(t) Since the MGF for a single normal variable is M(t)=eμt+(1/2)σ2t2M(t) = e^{\mu t + (1/2)\sigma^2 t^2}, the resulting MGF of the sum is: MX+Y(t)=eμ1t+(1/2)σ12t2⋅eμ2t+(1/2)σ22t2=e(μ1+μ2)t+(1/2)(σ12+σ22)t2M_{X+Y}(t) = e^{\mu_1 t + (1/2)\sigma_1^2 t^2} \cdot e^{\mu_2 t + (1/2)\sigma_2^2 t^2} = e^{(\mu_1 + \mu_2) t + (1/2)(\sigma_1^2 + \sigma_2^2) t^2}

By the uniqueness theorem, X+YX+Y must be distributed as Normal(μ1+μ2,σ12+σ22)\mathbf{\text{Normal}(\mu_1 + \mu_2, \sigma_1^2 + \sigma_2^2)}.

6.4 Bivariate Normal Distributions

This section discusses the properties of a joint distribution where linear combinations of the constituent variables are always normally distributed.

Definition and Properties

💡

Bivariate Normal

A pair of random variables (X,Y)(X, Y) is bivariate normal if every linear combination aX+bYaX + bY is a normally distributed random variable.

  1. Marginal Normality: XX and YY individually are also normally distributed.
  2. Determination by Moments: The joint distribution is completely determined by the means, variances, and correlation.
  3. Independence is Uncorrelation: For bivariate normal variables, Cov[X,Y]=0Cov[X, Y] = 0 if and only if XX and YY are independent.

Analogy: Summarizing a continuous random variable is like weighing an object. The Expected Value is the measurement of the object’s mass (its central tendency), while the Variance is a measure of the precision of the scale (how spread out the possible readings are). Covariance, then, is like simultaneously weighing two interconnected objects to see how much one influences the other’s reading.