4 Probability Rules and Distributions
4.1 Overview
This section marks the transition from describing observed data to reasoning about uncertain outcomes. In earlier sections, the focus was on concepts such as center, spread, shape, and trends. Now we extend that framework by asking: “What could happen, and how likely is each outcome?” Answering this requires more than descriptive statistics. It requires looking at probabilities and probability distributions.
Things to cover:
- Probability Rules
- Binomial Distribution
- Normal Distribution
- Poisson Distribution
- Exponential Distribution
4.2 Probability
4.2.1 Probability Terminology
A probability is a number from 0 to 1 describing how likely an event is.
| Term | Meaning | Example |
|---|---|---|
| Experiment | Random process | Send a marketing email |
| Outcome | One possible result | Customer clicks |
| Event | One or more outcomes you’re interested in | Customer engages |
| Sample Space | All possible outcomes | {Click, Open, Ignore, Unsubscribe} |
4.2.2 Basic Probability Rules
4.2.2.1 Complement Rule
The complement rule finds the probability that an event does not happen.
\[ P(A^c)=1-P(A) \] If event A happens with probability P(A), its complement, written A^c, means “not A.”
For example:
If there’s a 20% chance of clicking: \(P(\text{No click})=1-.20=.80\)
4.2.2.2 Addition Rule
The addition rule finds the probability that event A OR event B occurs.
\[P(A\cup B)=P(A)+P(B) - P(A)\]
However, that only works if Event A and Event B can occur simultaneously.
If A and B are Mututally Exclusive: \[P(A\cup B)=P(A)+P(B)\]
For example:
Suppose:
- \(P(\text{opens email})=0.50\)
- \(P(\text{clicks link})=0.20\)
- \(P(\text{opens AND clicks})=0.15\)
Then:
\(P(\text{opens OR clicks})=0.50+0.20-0.15=0.55\)
Note: if mutually exclusive: \(P(\text{opens OR clicks})=0.50+0.20=0.70\)
4.2.2.3 Multiplication
The multiplication rule finds the probability that event A AND event B occur.
If A and B are Dependent Events:
\[P(A\cap B)=P(A)\times P(B\mid A)\]
If A and B are Independent Events:
\[P(A\cap B)=P(A)\cdot P(B)\]
4.2.3 Conditional Probability
Conditional Probability is the probability an event occurs given another has occured (i.e. probability of Event A given Event B occurred)
\[P(A\mid B)\]
4.3 Distributions
4.3.1 Binomial Distribution
A binomial distribution models the number of successes in a fixed number trials, where each trial has two possible outcomes: success or failure. Each trial is independent and the probability of success remains constant.
Discrete
\[X\sim\text{Binomial}(n,p)\]
- \(X \rightarrow \text{a discrete random variable}\)
- \(n \rightarrow \text{number of independent trials}\)
- \(p \rightarrow \text{probability of success}\)
dbinom(k, n, p) probability exactly k successes pbinom(k, n, p) probability less than or equal to k successes
4.3.2 Normal Distribution
A normal distribution models a continuous random variable with values that are distributed symmetrically around a mean, with values closer to the mean occurring more frequently than values farther from the mean.
mean = median = mode
Continuous
\[X\sim\text{Normal}(\mu,\sigma)\]
- \(X \rightarrow \text{a continuous random variable}\)
- \(\mu \rightarrow \text{mean}\)
- \(\sigma \rightarrow \text{standard deviation}\)
dnorm(x, mu, sigma) height of the curve at 100, not probability (continuous variable) pnorm(x, mu, sigma) area to the left of 100, which is a probability
- Z-score
- How many standard deviations away from mean.
\(z=\frac{x-\mu}{\sigma}\)
Empirical Rule
- 68% are within one standard deviation away from mean
- 95% are within two standard deviations away from mean
- 99.7% are within two standard deviations away from mean
Three ways to assess whether data are approximately normally distributed
- Histogram: Shows the shape of the data (roughly symmetric, bell-shaped)
- Q-Q plot: Compares observed data to what we’d expect from a normal distribution (points roughly follow a straight line)
- Shapiro-Wilk test: Formal hypothesis test for normality (p > 0.05: do not reject normality)
4.3.3 Poisson Distribution
A Poisson distribution models the number of times an event occurs within a fixed interval of time.
expected number of events (\(\lambda\)) = mean = variance
Discrete
\[X\sim\text{Poisson}(\lambda)\]
- \(X \rightarrow \text{number of events in the interval}\)
- \(\lambda \rightarrow \text{average number of events per interval}\)
Function
Calculates
Meaning
dpois(k, lambda)exactly k eventsppois(k, lambda)k or fewer events
- Chi-Square Goodness-of-Fit Test
- A chi-square goodness-of-fit test compares the observed event counts to the expected counts under a Poisson distribution.
\(H_0\): The data are consistent with a Poisson distribution.
\(H_A\): The data are not consistent with a Poisson distribution.
\(p > 0.05\): Fail to reject \(H_0\) → data are reasonably consistent with Poisson.
\(p < 0.05\): Reject \(H_0\) → evidence that the data do not fit a Poisson distribution.
- Overdispersion Check
- For a Poisson distribution, the mean and variance should be approximately equal.
\[ \frac{\text{Variance}}{\text{Mean}} \approx 1 \]
4.3.4 Exponential Distribution
An exponential distribution models the time between events in a Poisson process. While the Poisson distribution models how many events occur in an interval, the exponential distribution models how long until the next event occurs.
Continuous
\[ T \sim \text{Exponential}(\lambda) \]
\(T \rightarrow \text{waiting time until the next event}\)
\(\lambda \rightarrow \text{average rate at which events occur}\)
dexp(t, rate = lambda)density at \(t\)pexp(t, rate = lambda)probability the event occurs within time \(t\)qexp(p, rate = lambda)waiting time associated with cumulative probability \(p\)rexp(n, rate = lambda)generate exponential waiting times
Exponential is memoryless: meaning independent events don’t remember what has already happened. What has occurred doesn’t impact what is to come.