Discrete Random Variables

Chapter 3 marks a critical shift in perspective from viewing outcomes as abstract events in a sample space to seeing them as numerical values generated by functions called “random variables” . This conceptual framework allows the use of mathematical analysis to describe and summarize the behavior of probabilistic experiments.

The chapter is structured around defining random variables, understanding how they transmit probability, classifying common distribution types, and examining the relationships between multiple variables.

3.1 Random Variables: Functions of Chance

In Chapter 1, we defined an experiment resulting in an outcome from a sample space $S$ . Often, we are interested not in the raw outcome (e.g., {Heads, Tails}) but in a numerical feature derived from it.

Concept Explanation: Random Variables

💡

Random Variable (X)

A Random Variable ( $X$ ) is simply a function that maps every outcome in the sample space $S$ to a real number.

A variable is specifically classified as discrete if its output values (its range, $T$ ) form a countable (or finite) subset of the real numbers.

Experiment	Sample Space ( $S$ )	Random Variable ( $X$ )	Range ( $T$ )
Flip coin 3 times	$\{hhh, hht, \dots, ttt\}$	Total number of heads	$\{0, 1, 2, 3\}$
Roll pair of dice	36 pairs (e.g., (1, 1))	Larger of the two values	$\{1, 2, 3, 4, 5, 6\}$

Concept: Probability Mass Function (PMF)

For a discrete random variable $X$ , its distribution is completely defined by knowing the probability associated with each possible value $t$ in its range $T$ .

PMF Definition

$f_X(t) = P(X = t)$

for all $t \in T$

How it works

The PMF assigns a probability “mass” to each discrete value.

Sum of all masses = 1.

$P(X \in A) = \sum_{t \in A} f_X(t)$

Example Question and Solution (PMF)

Coin Flip PMF

Question: If a coin is flipped three times, and $X$ is the total number of heads, find the PMF of $X$ .

View Detailed Solution ▼

Solution: The sample space $S$ has $|S|=8$ equally likely outcomes.

$P(X=0) = P(\{ttt\}) = 1/8$ .
$P(X=1) = P(\{htt, tht, tth\}) = 3/8$ .
$P(X=2) = P(\{hht, hth, thh\}) = 3/8$ .
$P(X=3) = P(\{hhh\}) = 1/8$ .

The PMF is $f_X(0)=1/8, f_X(1)=3/8, f_X(2)=3/8, f_X(3)=1/8$ .

Common Discrete Distributions

Distribution	Notation	PMF, $P(X=k)$	Context
Bernoulli	$X \sim Bernoulli(p)$	$p$ (if $k=1$ )	Single trial
Binomial	$X \sim Binomial(n, p)$	$\binom{n}{k} p^k (1-p)^{n-k}$	$k$ successes in $n$ trials
Geometric	$X \sim Geometric(p)$	$p (1-p)^{k-1}$	Trials until first success
Poisson	$X \sim Poisson(\lambda)$	$\frac{e^{-\lambda} \lambda^k}{k!}$	Rare events ( $n$ large, $p$ small)

3.2 Relationships Between Random Variables

When multiple random variables are defined on the same sample space, we analyze how they interact.

Concept: Independence

💡

Independence

Two random variables, $X$ and $Y$ , are independent if the occurrence of any event related to $X$ does not affect probabilities related to $Y$ .

$\mathbf{P(X=t, Y=u) = P(X=t) P(Y=u)}$

A sequence is i.i.d. (“independent and identically distributed”) if all variables are mutually independent and share the exact same distribution.

Joint & Conditional Distributions

The joint distribution captures the relationship between variables: $Q((a, b)) = P(X=a, Y=b)$ .

Example Question and Solution (Joint Table)

Joint Probability Table

Let $X$ and $Y$ be defined by the following joint distribution table. Find $P(X=1 | Y=0)$ .

	X=1	X=2	Total P(Y=b)
Y=0	1/4	1/8	3/8
Y=1	1/4	1/4	4/8
Y=2	0	1/8	1/8
Total	1/2	1/2	1

View Detailed Solution ▼

Solution: Using the formula for conditional probability: $P(X=1 | Y=0) = \frac{P(X=1, Y=0)}{P(Y=0)} = \frac{1/4}{3/8}$

$P(X=1 | Y=0) = \frac{1}{4} \times \frac{8}{3} = \frac{2}{3}$

Key Property: Memoryless Distribution

M ♦

Memoryless

Past failures don’t
affect future success.

M ♦

Geometric

💡

Memoryless Property

The Geometric distribution is memoryless. If $X$ is the number of trials until the first success:

$\mathbf{P(X > n+m | X > n) = P(X > m)}$

The sequence “starts over” every time a failure occurs.

3.3 Functions of Random Variables

A function of a random variable, $Z = f(X)$ , is itself a new random variable.

Convolution (Sum of Independent Variables)

When examining the sum of two independent random variables, $Z = X+Y$ , the distribution $P(Z=n)$ is calculated using convolution.

Convolution Sum

$P(Z=n) = \sum_{j=0}^{n} P(X=j) \cdot P(Y=n-j)$

Logic

We sum the probabilities of all possible ways to get a total of $n$ :

$X=0, Y=n$

$X=1, Y=n-1$

…etc…

Example Application (Sum of Poissons)

Sum of Poissons

Question: If $X \sim Poisson(\lambda_1)$ and $Y \sim Poisson(\lambda_2)$ are independent, what is the distribution of their sum $Z = X+Y$ ?

View Detailed Solution ▼

Using the convolution sum and algebraic manipulation (including the binomial expansion), it can be proven that the sum of two independent Poisson variables is also a Poisson variable:

$\mathbf{Z \sim Poisson(\lambda_1 + \lambda_2)}$

All Chapters in this Book

Lesson 1

Basic Concepts

Foundational mathematical framework for probability, including definitions, axioms, conditional probability, and Bayes' Theorem.

Lesson 2

Sampling and Repeated Trials

Models based on repeated independent trials, focusing on Bernoulli trials and sampling methods.

Lesson 3

Discrete Random Variables

Formalizing random variables, probability mass functions, and independence.

Lesson 4

Summarizing Discrete Random Variables

Deriving numerical characteristics—expected value, variance, and standard deviation—to summarize behavior of discrete random variables.

Lesson 5

Continuous Probabilities and Random Variables

Transitioning from discrete sums to continuous integrals, density functions, and key distributions like Normal and Exponential.

Lesson 6

Summarising Continuous Random Variables

Extending expected value and variance to continuous variables, exploring Moment Generating Functions and Bivariate Normal distributions.

Lesson 7

Sampling and Descriptive Statistics

Transitioning from probability to statistics: using sample data to estimate population parameters like mean and variance.

Lesson 8

Sampling Distributions and Limit Theorems

The theoretical foundations of inference: Joint Distributions, Weak Law of Large Numbers (WLLN), and geometrical convergence via the Central Limit Theorem (CLT).

Lesson 9

Estimation and Hypothesis Testing

The core of statistical inference: Method of Moments, Maximum Likelihood, Confidence Intervals, and Hypothesis Testing.

Lesson 10

Linear Regression

Modeling linear relationships, least squares, and regression inference.

Discrete Random Variables

Discrete Random Variables

3.1 Random Variables: Functions of Chance

Concept Explanation: Random Variables

Random Variable (X)

Concept: Probability Mass Function (PMF)

PMF Definition

How it works

Example Question and Solution (PMF)

Coin Flip PMF

Common Discrete Distributions

3.2 Relationships Between Random Variables

Concept: Independence

Independence

Joint & Conditional Distributions

Example Question and Solution (Joint Table)

Joint Probability Table

Key Property: Memoryless Distribution

Memoryless

Memoryless Property

3.3 Functions of Random Variables

Convolution (Sum of Independent Variables)

Convolution Sum

Logic

Example Application (Sum of Poissons)

Sum of Poissons

All Chapters in this Book

Basic Concepts

Sampling and Repeated Trials

Discrete Random Variables

Summarizing Discrete Random Variables

Continuous Probabilities and Random Variables

Summarising Continuous Random Variables

Sampling and Descriptive Statistics

Sampling Distributions and Limit Theorems

Estimation and Hypothesis Testing

Linear Regression

finding (solutions) x