Probability Distributions

In statistics, a probability distribution is a function that describes the likelihood of obtaining the possible values that a random variable can assume. In simpler terms, it tells us how likely different outcomes are. We will explore several important probability distributions that are fundamental to understanding and analyzing data, especially in fields like quality control, genetics, finance, and physics. These distributions help us model random phenomena and make predictions.

Binomial Distribution

The binomial distribution is used for experiments that have only two possible outcomes (success or failure) and are repeated a fixed number of times, with each trial being independent. Think of flipping a coin multiple times. Each flip is independent, and there are only two outcomes: heads (success) or tails (failure).

Conditions for Binomial Distribution:

• The experiment consists of a fixed number of trials, denoted by 'n'.
• Each trial has only two possible outcomes: success or failure.
• The probability of success, denoted by 'p', is the same for each trial.
• The probability of failure is 'q = 1 - p', which is also constant for each trial.
• The trials are independent of each other.

Probability Mass Function (PMF):

The probability of getting exactly 'k' successes in 'n' trials is given by the formula: P(X = k) = C(n, k) * p^k * q^(n-k) where C(n, k) is the binomial coefficient, calculated as n! / (k! * (n-k)!).

Mean and Variance:

• Mean (Expected Value): E(X) = n * p
• Variance: Var(X) = n * p * q

Example:

Suppose a fair coin is tossed 5 times. What is the probability of getting exactly 3 heads? Here, n = 5 (number of trials), k = 3 (number of successes/heads), p = 0.5 (probability of heads), and q = 1 - 0.5 = 0.5. P(X = 3) = C(5, 3) * (0.5)³ * (0.5)^(5-3) C(5, 3) = 5! / (3! * 2!) = (5 * 4) / (2 * 1) = 10 P(X = 3) = 10 * (0.125) * (0.25) = 10 * 0.03125 = 0.3125 So, the probability of getting exactly 3 heads in 5 tosses is 0.3125 or 31.25%.

Poisson Distribution

The Poisson distribution is used to model the probability of a given number of events occurring in a fixed interval of time or space, provided these events occur with a known constant mean rate and independently of the time since the last event. It's often used for rare events or events occurring at random points.

Conditions for Poisson Distribution:

• Events occur one at a time.
• The occurrence of an event does not affect the probability of another event occurring.
• The average rate (mean) of events per interval is constant.

Probability Mass Function (PMF):

The probability of observing exactly 'k' events in an interval is given by: P(X = k) = (λ^k * e^-λ) / k! where λ (lambda) is the average number of events in the interval, and 'e' is the base of the natural logarithm (approximately 2.71828).

Mean and Variance:

• Mean: E(X) = λ
• Variance: Var(X) = λ

Example:

A call center receives an average of 10 calls per hour. What is the probability of receiving exactly 7 calls in a given hour? Here, λ = 10, and k = 7. P(X = 7) = (10⁷ * e^-10) / 7! e^-10 ≈ 0.0000454 7! = 5040 P(X = 7) = (10,000,000 * 0.0000454) / 5040 ≈ 454 / 5040 ≈ 0.09008 The probability of receiving exactly 7 calls in an hour is approximately 0.09008 or 9.01%.

The Poisson distribution can be approximated by the binomial distribution when 'n' is very large and 'p' is very small, such that n*p = λ.

Normal Distribution

The normal distribution, also known as the Gaussian distribution or bell curve, is perhaps the most important probability distribution in statistics. It is a continuous probability distribution that is symmetric about its mean. Many natural phenomena, such as heights, weights, blood pressure, and measurement errors, tend to follow a normal distribution.

Characteristics:

• It is symmetric around the mean. The mean, median, and mode are all equal.
• The total area under the curve is 1.
• The curve is asymptotic to the x-axis, meaning it approaches the x-axis but never touches it.
• The shape of the curve is determined by its mean (μ) and standard deviation (σ).

Probability Density Function (PDF):

The PDF for a normal distribution is given by: f(x | μ, σ²) = (1 / (σ * sqrt(2π))) * e^{-((x - μ)2 / (2σ2))} where: - x is the variable - μ is the mean - σ is the standard deviation - σ² is the variance - π (pi) is approximately 3.14159 - e is the base of the natural logarithm (approximately 2.71828)

Standard Normal Distribution:

A special case of the normal distribution is the standard normal distribution, which has a mean (μ) of 0 and a standard deviation (σ) of 1. It is often denoted by Z. Any normal distribution can be converted to a standard normal distribution using the z-score formula: z = (x - μ) / σ

Empirical Rule (68-95-99.7 Rule):

For a normal distribution:

• Approximately 68% of the data falls within one standard deviation of the mean (μ ± σ).
• Approximately 95% of the data falls within two standard deviations of the mean (μ ± 2σ).
• Approximately 99.7% of the data falls within three standard deviations of the mean (μ ± 3σ).

Example:

The IQ scores of a population are normally distributed with a mean of 100 and a standard deviation of 15. What percentage of people have an IQ score between 85 and 115? Here, μ = 100, σ = 15. The range 85 to 115 is exactly μ ± σ (100 ± 15). According to the empirical rule, approximately 68% of the data falls within one standard deviation of the mean. So, about 68% of people have IQ scores between 85 and 115.

Gamma Distribution

The gamma distribution is a two-parameter continuous probability distribution. It is often used to model the waiting time until a certain number of events occur in a Poisson process. It is flexible and can take on various shapes depending on its parameters.

Parameters:

The gamma distribution is typically defined by two parameters:

• Shape parameter (k or α): This parameter determines the shape of the distribution.
• Rate parameter (β or θ): This parameter is the inverse of the scale parameter. Sometimes, a scale parameter (θ = 1/β) is used instead of the rate parameter.

Probability Density Function (PDF):

Using shape parameter 'k' and rate parameter 'β': f(x; k, β) = (β^k * x^k-1 * e^-βx) / Γ(k) for x > 0 where Γ(k) is the gamma function, defined as Γ(k) = ∫₀^∞ t^k-1 e^-t dt. For integer values of k, Γ(k) = (k-1)!.

Mean and Variance:

• Mean: E(X) = k / β
• Variance: Var(X) = k / β²

Relationship to other distributions:

• If k=1, the gamma distribution becomes the exponential distribution.
• The sum of 'k' independent exponential random variables with rate 'β' follows a gamma distribution with shape 'k' and rate 'β'.

Example:

Consider the time (in hours) until the 3rd customer arrives at a store, given that customers arrive according to a Poisson process with an average rate of 2 customers per hour. This waiting time follows a gamma distribution with shape k=3 and rate β=2. The average waiting time is E(X) = k/β = 3/2 = 1.5 hours. The variance of the waiting time is Var(X) = k/β² = 3/2² = 3/4 = 0.75 hours².

Beta Distribution

The beta distribution is a continuous probability distribution defined on the interval [0, 1]. It is often used to model probabilities or proportions, which by definition lie between 0 and 1. It is particularly useful in Bayesian statistics as a prior distribution for a probability parameter.

Parameters:

The beta distribution is defined by two positive shape parameters, α (alpha) and β (beta).

Probability Density Function (PDF):

f(x; α, β) = (x^α-1 * (1-x)^β-1) / B(α, β) for 0 ≤ x ≤ 1 where B(α, β) is the beta function, defined as B(α, β) = Γ(α)Γ(β) / Γ(α + β).

Mean and Variance:

• Mean: E(X) = α / (α + β)
• Variance: Var(X) = (αβ) / ((α + β)² * (α + β + 1))

Shape variations:

• If α = 1 and β = 1, it's a uniform distribution on [0, 1].
• If α > 1 and β > 1, the distribution is unimodal (has a single peak).
• If α < 1 and β < 1, the distribution is U-shaped.
• If α = β, the distribution is symmetric around 0.5.

Example:

Suppose we want to model the proportion of defective items produced by a machine. If we have prior belief that the proportion is likely around 0.1, and we have observed data that suggests a certain shape, we could use a beta distribution. For instance, a beta distribution with α=2 and β=18 would have a mean of 2/(2+18) = 2/20 = 0.1. The shape would be skewed towards 0, reflecting a belief that defects are rare.

Cauchy Distribution

The Cauchy distribution, also known as the Lorentz distribution, is a continuous probability distribution that is characterized by its heavy tails and lack of defined mean, variance, or higher moments. It arises in physics, particularly in spectroscopy, and in probability theory.

Parameters:

It has two parameters:

• Location parameter (x₀): This determines the center of the distribution.
• Scale parameter (γ): This determines the half-width at half-maximum (HWHM).

Probability Density Function (PDF):

f(x; x₀, γ) = 1 / (πγ * (1 + ((x - x₀) / γ)²))

Mean and Variance:

The mean and variance are undefined because the integrals required to calculate them do not converge. This means extreme values are much more likely than in a normal distribution.

Relationship to Normal Distribution:

The Cauchy distribution can be generated as the ratio of two independent standard normal random variables. If Z₁ ~ N(0,1) and Z₂ ~ N(0,1), then Z₁ / Z₂ follows a standard Cauchy distribution (x₀=0, γ=1).

Example:

In physics, the shape of a resonance curve in spectroscopy can often be described by a Cauchy distribution. The location parameter x₀ corresponds to the peak frequency, and the scale parameter γ relates to the width of the resonance. The undefined mean signifies that the peak might not be a good representation of the "average" frequency due to the heavy tails.

Multinomial Distribution

The multinomial distribution is a generalization of the binomial distribution. While the binomial distribution deals with the number of successes in a fixed number of independent trials with two outcomes, the multinomial distribution deals with the number of outcomes for each of several categories, when there are more than two possible outcomes for each trial.

Conditions:

• A fixed number of trials, 'n'.
• Each trial results in one of 'k' possible outcomes (categories).
• The probability of each outcome (p₁, p₂, ..., p_k) is constant for each trial, where Σp_i = 1.
• The trials are independent.

Probability Mass Function (PMF):

The probability of observing exactly x₁ outcomes of category 1, x₂ outcomes of category 2, ..., and x_k outcomes of category k, where Σx_i = n, is given by: P(X₁=x₁, ..., X_k=x_k) = (n! / (x₁! * x₂! * ... * x_k!)) * (p₁^x₁ * p₂^x₂ * ... * p_k^x_k)

Mean and Covariance:

• The marginal distribution of each X_i is binomial with parameters n and p_i.
• E(X_i) = n * p_i
• Cov(X_i, X_j) = -n * p_i * p_j for i ≠ j

Example:

Imagine rolling a fair six-sided die 10 times (n=10). The possible outcomes (categories) are 1, 2, 3, 4, 5, 6, each with probability p_i = 1/6. What is the probability of rolling three 1s, two 2s, one 3, zero 4s, two 5s, and two 6s? (x₁=3, x₂=2, x₃=1, x₄=0, x₅=2, x₆=2). P(...) = (10! / (3! * 2! * 1! * 0! * 2! * 2!)) * ((1/6)³ * (1/6)² * (1/6)¹ * (1/6)⁰ * (1/6)² * (1/6)²) Calculating this would give the specific probability.

Hypergeometric Distribution

The hypergeometric distribution is used when sampling without replacement from a finite population. It describes the probability of 'k' successes in 'n' draws, without replacement, from a finite population of size 'N' that contains exactly 'K' successes. This is in contrast to the binomial distribution, which assumes sampling with replacement or from an infinite population.

Conditions:

• The population is finite and consists of N items.
• These N items contain K items of interest (successes) and N-K items not of interest (failures).
• A sample of size n is drawn without replacement.

Probability Mass Function (PMF):

The probability of getting exactly 'k' successes in a sample of size 'n' is: P(X = k) = [C(K, k) * C(N-K, n-k)] / C(N, n) where: - C(a, b) is the combination "a choose b". - N is the population size. - K is the number of success states in the population. - n is the number of draws (sample size). - k is the number of observed successes in the sample.

Constraints:

• max(0, n - (N - K)) ≤ k ≤ min(n, K)

Example:

A bag contains 10 marbles: 6 red and 4 blue. If you draw 3 marbles without replacement, what is the probability that exactly 2 of them are red? Here, N = 10 (total marbles), K = 6 (red marbles), n = 3 (marbles drawn), k = 2 (red marbles drawn). P(X = 2) = [C(6, 2) * C(10-6, 3-2)] / C(10, 3) C(6, 2) = 6! / (2! * 4!) = (6 * 5) / (2 * 1) = 15 C(4, 1) = 4! / (1! * 3!) = 4 C(10, 3) = 10! / (3! * 7!) = (10 * 9 * 8) / (3 * 2 * 1) = 120 P(X = 2) = (15 * 4) / 120 = 60 / 120 = 0.5 The probability of drawing exactly 2 red marbles is 0.5 or 50%.

Negative Binomial Distribution

The negative binomial distribution describes the probability of the number of failures before a specified number of successes occurs in a sequence of independent Bernoulli trials. It's closely related to the binomial distribution but focuses on the number of trials needed to achieve a certain number of successes, rather than the number of successes in a fixed number of trials.

Definition 1 (Number of Failures):

Let 'r' be the desired number of successes, and 'p' be the probability of success in each trial. The probability of observing exactly 'k' failures before the r-th success is: P(X = k) = C(k + r - 1, r - 1) * p^r * (1-p)^k where k = 0, 1, 2, ...

Definition 2 (Number of Trials):

Alternatively, let 'r' be the desired number of successes, and 'p' be the probability of success. The probability that the r-th success occurs on the n-th trial is: P(Y = n) = C(n - 1, r - 1) * p^r * (1-p)^(n-r) where n = r, r+1, r+2, ... Note that Y = X + r (number of trials = number of failures + number of successes).

Mean and Variance (for number of failures 'k'):

• Mean: E(X) = r * (1-p) / p
• Variance: Var(X) = r * (1-p) / p²

Example:

A basketball player makes free throws with a probability of 0.7 (p=0.7). What is the probability that they miss 3 shots before making their 5th successful shot? Here, r = 5 (successes), k = 3 (failures), p = 0.7, (1-p) = 0.3. P(X = 3) = C(3 + 5 - 1, 5 - 1) * (0.7)⁵ * (0.3)³ P(X = 3) = C(7, 4) * (0.16807) * (0.027) C(7, 4) = 7! / (4! * 3!) = (7 * 6 * 5) / (3 * 2 * 1) = 35 P(X = 3) = 35 * 0.16807 * 0.027 ≈ 0.1585 The probability of missing 3 shots before making the 5th is approximately 0.1585 or 15.85%.