Statistical Methods: Population, Sample, Data Collection, Presentation, Measures of Location and Dispersion, Moments, and Cumulants

Population and Random Sample

In statistics, we often study a large group of individuals or items to understand certain characteristics. This entire group is called the population. For example, if we want to study the average height of all college students in a country, then all college students in that country form the population.

Sometimes, studying the entire population is impractical or impossible due to its size, cost, or time constraints. In such cases, we study a subset of the population, which is called a sample. A sample should be representative of the population so that the conclusions drawn from the sample can be generalized to the population.

A random sample is a sample where every individual or item in the population has an equal and independent chance of being selected. This randomness is crucial for ensuring that the sample is unbiased and representative.

There are two main types of random sampling:

• Simple Random Sampling (SRS): Every possible sample of a given size has an equal chance of being selected. This can be done with or without replacement.
• Stratified Random Sampling: The population is divided into subgroups (strata) based on some characteristic, and then a simple random sample is taken from each stratum. This is useful when the population is heterogeneous.
• Systematic Sampling: Every k-th item is selected from a list after a random start.
• Cluster Sampling: The population is divided into clusters, and then a random sample of clusters is selected. All individuals within the selected clusters are then studied.

Example: To estimate the average income of households in a city, the entire city's households are the population. If we randomly select 500 households to survey, these 500 households form a random sample. If we ensure each household had an equal chance of being chosen, it's a simple random sample. If we first divide the city into neighborhoods (strata) and then randomly pick households from each neighborhood, it's stratified sampling.

Data Collection and Presentation

Once we have decided on the sampling method, the next step is to collect data. Data can be collected through various methods like surveys, experiments, observations, or by using existing records. After collection, data needs to be organized and presented in a way that makes it easy to understand and analyze.

Types of Data

Data can be broadly classified into two types:

• Qualitative Data: Describes qualities or characteristics that cannot be measured numerically. Examples include gender, color, or opinion (e.g., 'satisfied' or 'unsatisfied').
• Quantitative Data: Data that can be measured numerically.
  • Discrete Data: Can only take specific values, usually integers. It arises from counting. Example: Number of cars owned by a family.
  • Continuous Data: Can take any value within a given range. It arises from measurement. Example: Height of a person.

Methods of Data Presentation

Raw data, especially from large samples, is often difficult to interpret. Presentation methods help in summarizing and visualizing the data.

1. Tabular Presentation: Data is organized in rows and columns.

• Frequency Distribution Table: This table shows how often each value or range of values appears in the data. For quantitative data, we often group values into classes or bins.

Example (Discrete Data): Suppose we survey 30 students about the number of siblings they have:
Data: 0, 1, 2, 1, 3, 0, 1, 2, 1, 0, 4, 1, 2, 1, 0, 1, 2, 3, 1, 0, 2, 1, 1, 0, 2, 1, 3, 0, 1, 2
Frequency Distribution:

Number of Siblings (x)	Frequency (f)
0	7
1	12
2	7
3	3
4	1
Total	30

Example (Continuous Data): Heights of 20 students (in cm):
155, 162, 158, 170, 165, 159, 168, 172, 160, 166, 157, 163, 169, 171, 164, 156, 167, 173, 161, 168
Grouped Frequency Distribution (Classes: 155-159, 160-164, etc.):

Height (cm)	Frequency (f)
155-159	5
160-164	5
165-169	6
170-174	4
Total	20

2. Graphical Presentation: Visual representation of data using charts and graphs.

• Bar Chart: Used for qualitative data or discrete quantitative data. Each bar represents a category, and its height corresponds to the frequency.
• Histogram: Used for continuous quantitative data, especially grouped frequency distributions. Bars are adjacent, representing intervals. The area of each bar is proportional to the frequency.
• Frequency Polygon: A line graph connecting the midpoints of the tops of the bars in a histogram. It's useful for comparing frequency distributions.
• Pie Chart: Represents data as slices of a circle, where each slice's proportion corresponds to the category's percentage of the total. Best for showing proportions of a whole.
• Ogive (Cumulative Frequency Curve): A graph of cumulative frequency distribution. It shows the number of observations below a certain value.

Measures of Location (Central Tendency)

Measures of central tendency describe the center or a typical value of a dataset. They give us a single value that represents the dataset. The most common measures are the mean, median, and mode.

1. Mean (Arithmetic Mean)

The mean is the sum of all values divided by the number of values. It's the most common measure of central tendency.

For a dataset $x_1, x_2, ..., x_n$: $$ \bar{x} = \frac{\sum_{i=1}^{n} x_i}{n} $$

For a frequency distribution where $x_i$ are values with frequencies $f_i$: $$ \bar{x} = \frac{\sum_{i=1}^{k} f_i x_i}{\sum_{i=1}^{k} f_i} = \frac{\sum f x}{N} $$ where $N = \sum f$ is the total number of observations.

Pros: Uses all data points, unique value.
Cons: Sensitive to outliers (extreme values).

Example: Data: 2, 4, 6, 8, 10
Mean = (2 + 4 + 6 + 8 + 10) / 5 = 30 / 5 = 6

2. Median

The median is the middle value of a dataset when arranged in ascending or descending order. If the number of observations (n) is odd, the median is the $((n+1)/2)^{th}$ value. If n is even, the median is the average of the $(n/2)^{th}$ and $(n/2 + 1)^{th}$ values.

For grouped data, the median class is the class where the cumulative frequency first exceeds $N/2$. The median can be calculated using the formula: $$ \text{Median} = L + \left( \frac{N/2 - CF}{f} \right) \times w $$ where:

• $L$ = Lower limit of the median class
• $N$ = Total frequency
• $CF$ = Cumulative frequency of the class preceding the median class
• $f$ = Frequency of the median class
• $w$ = Width of the median class

Pros: Not affected by outliers, easy to understand.
Cons: Does not use all data points, less stable than mean for large datasets.

Example: Data: 2, 4, 6, 8, 10 (n=5, odd)
Median = $((5+1)/2)^{th}$ value = 3rd value = 6
Data: 2, 4, 6, 8, 10, 12 (n=6, even)
Median = Average of $(6/2)^{th}$ and $(6/2+1)^{th}$ values = Average of 3rd and 4th values = (6 + 8) / 2 = 7

3. Mode

The mode is the value that appears most frequently in a dataset. A dataset can have one mode (unimodal), two modes (bimodal), or more (multimodal). If all values appear with the same frequency, there is no mode.

For grouped data, the mode class is the class with the highest frequency. The mode can be calculated using the formula: $$ \text{Mode} = L + \left( \frac{f_m - f_{m-1}}{(f_m - f_{m-1}) + (f_m - f_{m+1})} \right) \times w $$ where:

• $L$ = Lower limit of the modal class
• $f_m$ = Frequency of the modal class
• $f_{m-1}$ = Frequency of the class preceding the modal class
• $f_{m+1}$ = Frequency of the class succeeding the modal class
• $w$ = Width of the modal class

Pros: Easy to find for discrete data, useful for qualitative data.
Cons: May not exist, may not be unique, not always representative.

Example: Data: 2, 4, 6, 4, 8, 4, 10
Mode = 4 (appears 3 times)

Measures of Dispersion (Variability)

Measures of dispersion indicate how spread out or scattered the data points are from the center. They tell us about the variability in the data.

1. Range

The range is the simplest measure of dispersion. It is the difference between the highest and lowest values in the dataset. $$ \text{Range} = \text{Maximum Value} - \text{Minimum Value} $$

Pros: Easy to calculate and understand.
Cons: Highly affected by outliers, uses only two extreme values.

Example: Data: 2, 4, 6, 8, 10
Range = 10 - 2 = 8

2. Quartile Deviation (Semi-Interquartile Range)

Quartiles divide the data into four equal parts.

• $Q_1$ (First Quartile): The value below which 25% of the data falls.
• $Q_2$ (Second Quartile): The median, below which 50% of the data falls.
• $Q_3$ (Third Quartile): The value below which 75% of the data falls.

The Interquartile Range (IQR) is the difference between $Q_3$ and $Q_1$: $IQR = Q_3 - Q_1$. The Quartile Deviation (QD) is half of the IQR: $$ QD = \frac{Q_3 - Q_1}{2} $$

Pros: Not affected by extreme values.
Cons: Does not consider all data points.

Calculation for Ungrouped Data:

• Arrange data in ascending order.
• Calculate the position of $Q_1$ as $(n+1)/4$.
• Calculate the position of $Q_3$ as $3(n+1)/4$.
• Find the values at these positions.

Calculation for Grouped Data: Similar to median calculation, but using cumulative frequencies relative to $N/4$ for $Q_1$ and $3N/4$ for $Q_3$. $$ Q_1 = L_1 + \left( \frac{N/4 - CF_1}{f_1} \right) \times w_1 $$ $$ Q_3 = L_3 + \left( \frac{3N/4 - CF_3}{f_3} \right) \times w_3 $$

3. Standard Deviation (σ for population, s for sample)

The standard deviation is the most widely used measure of dispersion. It measures the average amount by which data points deviate from the mean. It is the square root of the variance.

Population Standard Deviation (σ): $$ \sigma = \sqrt{\frac{\sum_{i=1}^{N} (x_i - \mu)^2}{N}} $$ where $\mu$ is the population mean.

Sample Standard Deviation (s): $$ s = \sqrt{\frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n-1}} $$ where $\bar{x}$ is the sample mean. The use of $(n-1)$ in the denominator (Bessel's correction) provides a better, unbiased estimate of the population variance.

For a frequency distribution: $$ \sigma = \sqrt{\frac{\sum f (x_i - \mu)^2}{N}} \quad \text{or} \quad s = \sqrt{\frac{\sum f (x_i - \bar{x})^2}{n-1}} $$ A computationally simpler formula for variance ($s^2$) is: $$ s^2 = \frac{\sum f x^2 - (\sum f x)^2 / n}{n-1} $$

Pros: Uses all data points, mathematically tractable, widely used.
Cons: Sensitive to outliers.

Example: Data: 2, 4, 6, 8, 10. Mean ($\bar{x}$) = 6.
Deviations from mean: (2-6)=-4, (4-6)=-2, (6-6)=0, (8-6)=2, (10-6)=4
Squared deviations: 16, 4, 0, 4, 16
Sum of squared deviations = 16 + 4 + 0 + 4 + 16 = 40
Sample Variance ($s^2$) = 40 / (5-1) = 40 / 4 = 10
Sample Standard Deviation ($s$) = $\sqrt{10} \approx 3.16$

4. Variance

Variance is the average of the squared differences from the mean. It is the square of the standard deviation.

Population Variance ($\sigma^2$): $\sigma^2 = \frac{\sum (x_i - \mu)^2}{N}$
Sample Variance ($s^2$): $s^2 = \frac{\sum (x_i - \bar{x})^2}{n-1}$

Variance is measured in squared units, which can be difficult to interpret. Standard deviation, being the square root, is in the original units of the data.

Moments

Moments are mathematical tools used to describe the shape of a distribution. They are generalizations of the mean, variance, and skewness. Moments are typically calculated about the origin (raw moments) or about the mean (central moments).

Raw Moments (Moments about the Origin)

The k-th raw moment is denoted by $\mu'_k$ and is defined as: $$ \mu'_k = E[X^k] = \frac{\sum_{i=1}^{N} x_i^k}{N} $$ For a frequency distribution: $$ \mu'_k = \frac{\sum f x_i^k}{N} $$

• The 1st raw moment ($k=1$) is the mean: $\mu'_1 = E[X] = \mu$.
• The 2nd raw moment ($k=2$) is related to variance: $\mu'_2 = E[X^2]$.

Central Moments (Moments about the Mean)

The k-th central moment is denoted by $\mu_k$ and is defined as: $$ \mu_k = E[(X - \mu)^k] = \frac{\sum_{i=1}^{N} (x_i - \mu)^k}{N} $$ For a frequency distribution: $$ \mu_k = \frac{\sum f (x_i - \mu)^k}{N} $$

• The 1st central moment ($k=1$) is always zero: $\mu_1 = E[X - \mu] = \mu - \mu = 0$.
• The 2nd central moment ($k=2$) is the variance: $\mu_2 = E[(X - \mu)^2] = \sigma^2$.
• The 3rd central moment ($k=3$) is related to skewness.
• The 4th central moment ($k=4$) is related to kurtosis.

Relationship between Raw and Central Moments: Central moments can be expressed in terms of raw moments. For example:

• $\mu_1 = 0$
• $\mu_2 = \mu'_2 - (\mu'_1)^2$ (This is the variance formula using raw moments)
• $\mu_3 = \mu'_3 - 3\mu'_2 \mu'_1 + 2(\mu'_1)^3$
• $\mu_4 = \mu'_4 - 4\mu'_3 \mu'_1 + 6\mu'_2 (\mu'_1)^2 - 3(\mu'_1)^4$

Sheppard's Correction

Sheppard's correction is used when calculating moments for grouped continuous data, specifically when the data has been grouped into intervals of equal width ($w$). When data is grouped, some information about the exact values within each interval is lost. This grouping can lead to a slight underestimation of the variance (and higher moments). Sheppard's correction adjusts for this loss of information.

The correction is applied to the second central moment (variance) and potentially higher central moments.

Corrected Variance ($\sigma^2_{corrected}$ or $s^2_{corrected}$): $$ \sigma^2_{corrected} = \sigma^2_{uncorrected} - \frac{w^2}{12} $$ where:

• $\sigma^2_{uncorrected}$ is the calculated variance from the grouped data (using the formula $\frac{\sum f (x_i - \bar{x})^2}{N}$ or $\frac{\sum f x_i^2 - (\sum f x_i)^2 / N}{N}$). Note: For sample variance, the denominator is N, not N-1, when applying Sheppard's correction directly to the formula based on N.
• $w$ is the width of the class interval.

For sample variance ($s^2$), the formula is typically applied to the population variance estimate first, or derived carefully. A common approach is to use the uncorrected variance calculated with N in the denominator: $$ s^2_{corrected} = s^2_{uncorrected} - \frac{w^2}{12} $$ where $s^2_{uncorrected} = \frac{\sum f (x_i - \bar{x})^2}{N}$.

When to Use: Sheppard's correction is appropriate for continuous data grouped into equal intervals, provided the tails of the distribution are negligible (i.e., the data doesn't extend much beyond the first and last class limits). It's less effective if the distribution is highly skewed or has heavy tails.

Example: Suppose we calculate the variance of grouped data to be 25, and the class interval width ($w$) is 5.
Sheppard's correction for variance = $w^2 / 12 = 5^2 / 12 = 25 / 12 \approx 2.08$.
Corrected Variance = $25 - 2.08 = 22.92$.

Sheppard's correction can also be extended to higher moments, but it becomes more complex. The correction for the fourth central moment involves terms like $w^4$.

Cumulants

Cumulants are another set of statistical measures that describe the shape of a distribution. They are related to moments but have certain additive properties that make them useful, especially in the study of sums of independent random variables.

Let $K(t)$ be the cumulant-generating function (CGF) of a random variable $X$. The CGF is the logarithm of the moment-generating function (MGF), $M(t) = E[e^{tX}]$. $$ K(t) = \log(M(t)) = \log(E[e^{tX}]) $$

The k-th cumulant, denoted by $\kappa_k$, is the coefficient of $t^k/k!$ in the Taylor series expansion of $K(t)$ about $t=0$: $$ K(t) = \sum_{k=1}^{\infty} \kappa_k \frac{t^k}{k!} $$

The first few cumulants are related to the moments as follows:

• 1st Cumulant ($\kappa_1$): This is equal to the first moment (mean). $$ \kappa_1 = E[X] = \mu $$
• 2nd Cumulant ($\kappa_2$): This is equal to the second central moment (variance). $$ \kappa_2 = Var(X) = \sigma^2 $$
• 3rd Cumulant ($\kappa_3$): This is equal to the third central moment ($\mu_3$). It is related to skewness. $$ \kappa_3 = \mu_3 $$
• 4th Cumulant ($\kappa_4$): This is related to the fourth central moment ($\mu_4$) and the second central moment ($\mu_2$). $$ \kappa_4 = \mu_4 - 3\mu_2^2 $$ This is related to kurtosis.

Properties of Cumulants

The key advantage of cumulants lies in their additive property for sums of independent random variables. If $X$ and $Y$ are independent random variables with cumulants $\kappa_k(X)$ and $\kappa_k(Y)$ respectively, then the cumulants of their sum $Z = X + Y$ are: $$ \kappa_k(Z) = \kappa_k(X) + \kappa_k(Y) $$ This property is extremely useful in probability theory and statistics, particularly in limit theorems. For instance, the variance of a sum of independent random variables is the sum of their variances, which follows directly from $\kappa_2(X+Y) = \kappa_2(X) + \kappa_2(Y)$.

Relationship with Moments: While moments describe the distribution directly, cumulants provide a different perspective, particularly useful for understanding the behavior of sums. The relationship between cumulants and moments can be derived from the definition of the CGF and MGF.

In summary, this unit covers the foundational concepts of statistical data handling, from defining populations and samples to presenting data and characterizing its central tendency and spread. Moments and cumulants provide deeper insights into the shape and properties of distributions, with Sheppard's correction addressing potential biases from data grouping.