Hypothesis Testing: z-test, t-test, ANOVA, Chi-Square, Mann-Whitney, Kruskal-Wallis, Rank Correlation

Introduction to Hypothesis Testing

Hypothesis testing is a fundamental statistical method used to make decisions or draw conclusions about a population based on sample data. It's a systematic procedure that helps us determine whether there is enough evidence in a sample to reject a specific statement (hypothesis) about the population. This process is crucial in research across various fields, including commerce, to validate theories, test the effectiveness of strategies, and understand relationships between variables.

The core idea is to formulate two competing hypotheses: the null hypothesis (H₀) and the alternative hypothesis (H₁). The null hypothesis represents the status quo or the statement of no effect, no difference, or no relationship. The alternative hypothesis is what we suspect might be true, representing an effect, a difference, or a relationship. We then collect sample data and use statistical tests to assess the likelihood of observing such data if the null hypothesis were true. If the probability is very low, we reject the null hypothesis in favor of the alternative.

Key Concepts in Hypothesis Testing

Before diving into specific tests, let's clarify some essential terms:

• Null Hypothesis (H₀): A statement of no significance, no difference, or no effect. It's the hypothesis we aim to disprove.
• Alternative Hypothesis (H₁ or Hₐ): A statement that contradicts the null hypothesis, suggesting there is a significant effect, difference, or relationship.
• Significance Level (α): The probability of rejecting the null hypothesis when it is actually true (Type I error). Commonly set at 0.05 (5%) or 0.01 (1%).
• p-value: The probability of obtaining test results at least as extreme as the results actually observed, assuming that the null hypothesis is true. A small p-value (typically ≤ α) provides evidence against the null hypothesis.
• Type I Error: Rejecting a true null hypothesis. The probability of this error is denoted by α.
• Type II Error: Failing to reject a false null hypothesis. The probability of this error is denoted by β.
• Critical Region: The set of values for the test statistic that leads to the rejection of the null hypothesis.
• Test Statistic: A value calculated from sample data that is used to decide whether to reject the null hypothesis.

Parametric vs. Non-Parametric Tests

Hypothesis tests can be broadly categorized into parametric and non-parametric tests. The choice between them depends on the nature of the data and the assumptions made about the population distribution.

• Parametric Tests: These tests assume that the data are drawn from a population that follows a specific probability distribution, usually the normal distribution. They also assume that the population variance is known or can be estimated. Examples include z-test, t-test, and ANOVA. Parametric tests are generally more powerful than non-parametric tests when their assumptions are met.
• Non-Parametric Tests: These tests do not require assumptions about the population distribution. They are often used when the data are ordinal, nominal, or when the assumptions of parametric tests are violated (e.g., small sample size, skewed distribution). They are sometimes called "distribution-free" tests. Examples include Chi-square test, Mann-Whitney U test, and Kruskal-Wallis test.

Parametric Tests

1. z-test

The z-test is a statistical hypothesis test used to determine whether a sample mean is statistically different from a known population mean, or to compare the means of two samples, when the population standard deviations are known. It assumes that the data are normally distributed or that the sample size is large (typically n > 30) due to the Central Limit Theorem.

Types of z-tests:

• One-sample z-test: Compares a sample mean (x̄) to a known population mean (μ₀).
• Two-sample z-test: Compares the means of two independent samples (x̄₁ and x̄₂) to determine if they are significantly different from each other. This test requires that the population standard deviations (σ₁ and σ₂) are known.

Formula for One-Sample z-test:

z = (x̄ - μ₀) / (σ / √n)

Where:

• x̄ = Sample mean
• μ₀ = Hypothesized population mean
• σ = Population standard deviation
• n = Sample size

Formula for Two-Sample z-test (Independent Samples):

z = (x̄₁ - x̄₂) / √[(σ₁²/n₁) + (σ₂²/n₂)]

Where:

• x̄₁ and x̄₂ = Sample means
• σ₁ and σ₂ = Population standard deviations
• n₁ and n₂ = Sample sizes

When to Use z-test:

• Population standard deviation is known.
• Sample size is large (n > 30), or the population is normally distributed.
• Comparing a sample mean to a population mean, or two sample means.

2. t-test (Student's t-test)

The t-test is used when the population standard deviation is unknown, and the sample size is small (n < 30). It assumes that the data are drawn from a normally distributed population. The t-test uses the sample standard deviation as an estimate of the population standard deviation. The test statistic follows a t-distribution, which is similar to the normal distribution but has heavier tails, accounting for the increased uncertainty due to estimating the population standard deviation.

Types of t-tests:

• One-sample t-test: Compares a sample mean (x̄) to a known population mean (μ₀) when σ is unknown.
• Independent samples t-test: Compares the means of two independent groups when σ₁ and σ₂ are unknown. Assumes equal variances (pooled t-test) or unequal variances (Welch's t-test).
• Paired samples t-test: Compares the means of two related groups (e.g., before-and-after measurements on the same subjects).

Formula for One-Sample t-test:

t = (x̄ - μ₀) / (s / √n)

Where:

• x̄ = Sample mean
• μ₀ = Hypothesized population mean
• s = Sample standard deviation
• n = Sample size

Degrees of freedom (df) for one-sample t-test = n - 1.

Formula for Independent Samples t-test (assuming equal variances):

t = (x̄₁ - x̄₂) / [sₚ * √(1/n₁ + 1/n₂)]

Where sₚ is the pooled standard deviation:

sₚ² = [ (n₁ - 1)s₁² + (n₂ - 1)s₂² ] / (n₁ + n₂ - 2)

Degrees of freedom (df) = n₁ + n₂ - 2.

Formula for Paired Samples t-test:

t = d̄ / (s<0xE1><0xB5><0xA2> / √n)

Where:

• d̄ = Mean of the differences between paired observations
• s<0xE1><0xB5><0xA2> = Standard deviation of the differences
• n = Number of pairs

Degrees of freedom (df) = n - 1 (where n is the number of pairs).

3. ANOVA (Analysis of Variance)

ANOVA is a statistical test used to compare the means of three or more independent groups. It determines whether there are any statistically significant differences between the means of these groups. Instead of performing multiple t-tests (which increases the risk of Type I error), ANOVA tests the overall hypothesis that all group means are equal.

The fundamental principle of ANOVA is to partition the total variation observed in the data into different sources of variation. It compares the variance between the groups (due to the treatment or independent variable) with the variance within the groups (due to random error).

Key Concepts in ANOVA:

• Independent Variable (Factor): The variable that defines the groups being compared (e.g., different teaching methods, different marketing strategies).
• Dependent Variable: The outcome variable being measured (e.g., test scores, sales figures).
• Sum of Squares (SS): Measures the total variation in the data.
• Mean Square (MS): Sum of squares divided by its corresponding degrees of freedom. Represents variance.
• F-statistic: The ratio of the variance between groups to the variance within groups (MS_between / MS_within). A large F-statistic suggests that the group means are significantly different.

Types of ANOVA:

• One-Way ANOVA: Used when there is one independent variable with three or more levels (groups).
• Two-Way ANOVA (and Factorial ANOVA): Used when there are two or more independent variables. It allows us to examine the main effects of each independent variable and their interaction effects.

Formula for One-Way ANOVA (F-statistic):

F = MS_between / MS_within

Where:

• MS_between = SS_between / df_between
• MS_within = SS_within / df_within
• SS_between = Σ nᵢ (x̄ᵢ - x̄<0xE1><0xB5><0xA1>)² (variation between group means and the overall mean)
• SS_within = Σ Σ (xᵢⱼ - x̄ᵢ)² (variation within each group around its group mean)
• df_between = k - 1 (k = number of groups)
• df_within = N - k (N = total number of observations)

Hypotheses for ANOVA:

• H₀: μ₁ = μ₂ = μ₃ = ... = μ<0xE2><0x82><0x96> (All group means are equal)
• H₁: At least one group mean is different from the others.

Post-hoc Tests:

If the ANOVA test results in rejecting the null hypothesis (i.e., there is a significant difference between at least two group means), post-hoc tests (like Tukey's HSD, Bonferroni) are used to determine which specific pairs of group means are significantly different from each other. These tests control the overall error rate.

Non-Parametric Tests

4. Chi-Square (χ²) Test

The Chi-Square test is a non-parametric statistical test used to analyze categorical data. It is used for two main purposes:

1. Goodness-of-Fit Test: To determine if a sample distribution matches a hypothesized population distribution.
2. Test of Independence: To determine if there is a significant association between two categorical variables in a population.

Chi-Square Goodness-of-Fit Test:

This test compares the observed frequencies in each category of a single categorical variable to the expected frequencies, assuming a specific distribution. It's useful for checking if observed data fits a theoretical distribution (e.g., uniform, normal, or a known population proportion).

Formula for Chi-Square Goodness-of-Fit Test:

χ² = Σ [(Oᵢ - Eᵢ)² / Eᵢ]

Where:

• Oᵢ = Observed frequency in category i
• Eᵢ = Expected frequency in category i
• Σ = Summation over all categories

Degrees of freedom (df) = k - 1 - p, where k is the number of categories and p is the number of parameters estimated from the data.

Chi-Square Test of Independence:

This test examines whether two categorical variables are associated or independent. It compares the observed frequencies in a contingency table (cross-tabulation of the two variables) with the frequencies that would be expected if the variables were independent.

Formula for Chi-Square Test of Independence:

χ² = Σ [(Oᵢⱼ - Eᵢⱼ)² / Eᵢⱼ]

Where:

• Oᵢⱼ = Observed frequency in cell (i, j) of the contingency table
• Eᵢⱼ = Expected frequency in cell (i, j)

The expected frequency Eᵢⱼ is calculated as: Eᵢⱼ = (Row Total * Column Total) / Grand Total.

Degrees of freedom (df) = (rows - 1) * (columns - 1).

5. Mann-Whitney U Test (Wilcoxon Rank-Sum Test)

The Mann-Whitney U test is a non-parametric alternative to the independent samples t-test. It is used to compare two independent groups when the assumptions of the t-test are not met (e.g., data are not normally distributed, or data are ordinal).

This test determines whether it is equally likely that a randomly selected value from one population will be less than or greater than a randomly selected value from a second population. It works by ranking all the data from both groups combined and then comparing the sum of the ranks for each group.

Hypotheses for Mann-Whitney U Test:

• H₀: The distributions of the two independent samples are identical. (Or, the median of one group is equal to the median of the other group).
• H₁: The distributions of the two independent samples are different. (Or, the median of one group is different from the other).

Procedure:

1. Combine all observations from both groups.
2. Rank all observations from smallest to largest. Assign average ranks in case of ties.
3. Calculate the sum of ranks for each group (R₁ and R₂).
4. Calculate the U statistic for each group using the formulas:
   • U₁ = n₁(n₂ + 1)/2 - R₁
   • U₂ = n₁(n₂ + 1)/2 - R₂
   (Where n₁ and n₂ are the sample sizes of group 1 and group 2, respectively).
5. The test statistic U is the smaller of U₁ and U₂.
6. Compare the calculated U statistic to a critical value from the Mann-Whitney U distribution table based on n₁ and n₂ and the chosen significance level (α). For larger sample sizes, the U statistic can be approximated by a normal distribution.

6. Kruskal-Wallis H Test

The Kruskal-Wallis H test is a non-parametric alternative to the one-way ANOVA. It is used to compare three or more independent groups when the assumptions of ANOVA are not met. It tests whether the medians of the groups are equal.

Similar to the Mann-Whitney U test, it ranks all the data from all groups combined and then compares the sum of ranks for each group.

Hypotheses for Kruskal-Wallis H Test:

• H₀: The distributions of all k independent samples are identical. (Or, the medians of all groups are equal).
• H₁: At least one group's distribution is different from the others. (Or, at least one group's median is different).

Procedure:

1. Combine all observations from all k groups.
2. Rank all observations from smallest to largest. Assign average ranks in case of ties.
3. Calculate the sum of ranks for each group (R₁, R₂, ..., R<0xE2><0x82><0x96>).
4. Calculate the H statistic:

   H = [12 / (N(N + 1))] * Σ (Rᵢ² / nᵢ) - 3(N + 1)

   Where:

   • N = Total number of observations across all groups
   • nᵢ = Sample size of group i
   • Rᵢ = Sum of ranks for group i
   • Σ = Summation over all k groups
5. If there are tied ranks, a correction factor can be applied to H, but for most practical purposes, the uncorrected H is used, especially with larger sample sizes.
6. Compare the calculated H statistic to a critical value from the Chi-Square distribution table with df = k - 1 (where k is the number of groups). For larger sample sizes, the H statistic approximates a Chi-Square distribution.

7. Rank Correlation (Spearman's Rank Correlation Coefficient)

Spearman's rank correlation coefficient (ρ or r<0xE2><0x82><0x9B>) is a non-parametric measure of the strength and direction of the monotonic relationship between two ranked variables. It assesses how well the relationship between two variables can be described using a monotonic function. A monotonic relationship is one where as one variable increases, the other variable tends to increase (positive monotonic) or decrease (negative monotonic), but not necessarily at a constant rate.

It is the non-parametric equivalent of Pearson's correlation coefficient. It is used when the assumptions of Pearson's correlation are not met (e.g., data are ordinal, or the relationship is not strictly linear but monotonic) or when outliers might unduly influence Pearson's r.

Hypotheses for Spearman's Rank Correlation:

• H₀: There is no monotonic relationship between the two variables (ρ = 0).
• H₁: There is a monotonic relationship between the two variables (ρ ≠ 0).

Formula:

Spearman's ρ is calculated using the same formula as Pearson's r, but applied to the ranks of the data instead of the raw scores.

ρ = 1 - [ 6 Σ dᵢ² ] / [ n(n² - 1) ]

Where:

• dᵢ = the difference between the ranks of the corresponding pairs of observations (rank of X - rank of Y)
• n = the number of pairs of observations
• Σ dᵢ² = the sum of the squared differences in ranks

If there are tied ranks, a correction is needed, or Pearson's r can be calculated on the ranks directly.

Interpretation:

• The value of ρ ranges from -1 to +1.
• +1 indicates a perfect positive monotonic relationship.
• -1 indicates a perfect negative monotonic relationship.
• 0 indicates no monotonic relationship.

Choosing the Right Test

Selecting the appropriate hypothesis test is crucial for valid statistical analysis. Here's a general guide:

Example Scenario

A marketing manager wants to test if a new advertising campaign has increased sales. They have sales data from 50 stores before the campaign and sales data from the same 50 stores after the campaign.

• Variables: Sales (continuous).
• Groups: Before campaign (n=50), After campaign (n=50).
• Relationship: Comparing means of two related groups (paired data).
• Assumptions: We would check if the differences in sales are normally distributed.

Choice of Test:

• If the differences are normally distributed, a **paired samples t-test** is appropriate.
• If the differences are not normally distributed, a non-parametric alternative like the **Wilcoxon signed-rank test** (not detailed here) would be used.

Another scenario: A researcher wants to know if there's a relationship between the type of industry (e.g., IT, Manufacturing, Services) and employee satisfaction level (e.g., Low, Medium, High).

• Variables: Industry Type (categorical), Satisfaction Level (ordinal/categorical).
• Relationship: Association between two categorical variables.

Choice of Test: A **Chi-Square test of independence** would be suitable here.