Correlation – Pearson and Spearman

Introduction to Correlation

Correlation is a statistical measure that describes the extent to which two variables change together. In simpler terms, it tells us if there's a relationship between two sets of data and how strong that relationship is. For instance, we might want to know if there's a correlation between the amount of fertilizer used on a plant and its height, or between a student's study hours and their exam scores.

When two variables are correlated, a change in one variable is associated with a change in the other. This association can be positive, negative, or nonexistent.

* Positive Correlation: Both variables tend to increase or decrease together. For example, as study hours increase, exam scores tend to increase. * Negative Correlation: As one variable increases, the other tends to decrease. For example, as the number of hours spent playing video games increases, exam scores might decrease. * Zero Correlation: There is no apparent relationship between the two variables. A change in one variable does not predict a change in the other.

The strength of the correlation is indicated by the correlation coefficient, a value that ranges from -1 to +1.

* A coefficient of +1 indicates a perfect positive linear relationship. * A coefficient of -1 indicates a perfect negative linear relationship. * A coefficient of 0 indicates no linear relationship.

Pearson Correlation Coefficient (r)

The Pearson correlation coefficient, often denoted by 'r', is the most common method used to measure the strength and direction of a *linear* relationship between two *continuous* variables. It assumes that the data is approximately normally distributed and that the relationship between the variables is linear.

Formula for Pearson Correlation Coefficient

The formula for the Pearson correlation coefficient (r) between two variables X and Y is:

$r = \frac{ \sum{(x_i - \bar{x})(y_i - \bar{y})} }{ \sqrt{\sum{(x_i - \bar{x})^2} \sum{(y_i - \bar{y})^2}} }$

Where:

• $x_i$ and $y_i$ are the individual data points for variables X and Y.
• $\bar{x}$ and $\bar{y}$ are the means of variables X and Y, respectively.
• $\sum$ denotes the sum of the values.

An alternative formula that is often easier for calculations is:

$r = \frac{ n(\sum{xy}) - (\sum{x})(\sum{y}) }{ \sqrt{[n\sum{x^2} - (\sum{x})^2][n\sum{y^2} - (\sum{y})^2]} }$

Where:

• $n$ is the number of data pairs.
• $\sum{xy}$ is the sum of the products of paired scores.
• $\sum{x}$ and $\sum{y}$ are the sums of the scores for each variable.
• $\sum{x^2}$ and $\sum{y^2}$ are the sums of the squared scores for each variable.

Steps to Calculate Pearson's r

1. List the paired data points for the two variables (X and Y).
2. Calculate the sum of X scores ($\sum{x}$) and the sum of Y scores ($\sum{y}$).
3. Calculate the sum of the squared X scores ($\sum{x^2}$) and the sum of the squared Y scores ($\sum{y^2}$).
4. Calculate the sum of the products of the paired scores ($\sum{xy}$).
5. Count the number of data pairs (n).
6. Plug these values into the formula: $r = \frac{ n(\sum{xy}) - (\sum{x})(\sum{y}) }{ \sqrt{[n\sum{x^2} - (\sum{x})^2][n\sum{y^2} - (\sum{y})^2]} }$
7. Calculate the value of r.

Interpretation of Pearson's r

The value of 'r' helps us understand the relationship:

Example of Pearson Correlation

Let's say we want to find the correlation between hours studied (X) and exam scores (Y) for 5 students:

Here, $n=5$.

Using the formula:

$r = \frac{ n(\sum{xy}) - (\sum{x})(\sum{y}) }{ \sqrt{[n\sum{x^2} - (\sum{x})^2][n\sum{y^2} - (\sum{y})^2]} }$

$r = \frac{ 5(2145) - (26)(395) }{ \sqrt{[5(158) - (26)^2][5(31575) - (395)^2]} }$

$r = \frac{ 10725 - 10270 }{ \sqrt{[790 - 676][157875 - 156025]} }$

$r = \frac{ 455 }{ \sqrt{[114][1850]} }$

$r = \frac{ 455 }{ \sqrt{210900} }$

$r = \frac{ 455 }{ 459.24 } \approx 0.99$

This high positive value (0.99) indicates a very strong positive linear relationship between hours studied and exam scores for this group of students.

Assumptions and Limitations of Pearson Correlation

Pearson's r relies on several assumptions:

• Linearity: The relationship between the two variables must be linear. If the relationship is curved, Pearson's r may underestimate or misrepresent the strength of the association.
• Normality: Both variables should be approximately normally distributed.
• Homoscedasticity: The spread of data points around the regression line should be roughly constant across all values of the predictor variable.
• No Outliers: Extreme values (outliers) can disproportionately influence the correlation coefficient.
• Continuous Variables: Both variables must be measured on an interval or ratio scale (i.e., they are continuous).

When these assumptions are violated, especially linearity, Pearson's r might not be the most appropriate measure.

Spearman Rank Correlation Coefficient (ρ or $r_s$)

The Spearman rank correlation coefficient, denoted by the Greek letter rho (ρ) or $r_s$, is a non-parametric measure of the strength and direction of a *monotonic* relationship between two ranked variables. A monotonic relationship is one where as one variable increases, the other variable consistently increases or consistently decreases, but not necessarily at a constant rate (i.e., not necessarily linear).

Spearman's rho is particularly useful when:

• The data is ordinal (ranked).
• The data is continuous but does not meet the assumptions of Pearson's r (e.g., not normally distributed, or the relationship is non-linear but monotonic).
• There are outliers that might unduly influence Pearson's r.

Formula for Spearman Rank Correlation Coefficient

Spearman's rho is calculated using the Pearson correlation formula, but applied to the *ranks* of the data rather than the raw scores.

If there are no tied ranks, the formula simplifies to:

$ρ = 1 - \frac{ 6 \sum{d_i^2} }{ n(n^2 - 1) }$

Where:

• $d_i$ is the difference between the ranks of each pair of observations ($rank(x_i) - rank(y_i)$).
• $n$ is the number of data pairs.
• $\sum{d_i^2}$ is the sum of the squared differences in ranks.

Steps to Calculate Spearman's ρ

1. List the paired data points for the two variables (X and Y).
2. Rank the data for each variable separately. Assign the lowest score a rank of 1, the next lowest a rank of 2, and so on.
3. Handling Ties: If there are tied scores for a variable, assign each tied score the *average* of the ranks they would have occupied. For example, if two scores would have been ranks 3 and 4, they both get a rank of (3+4)/2 = 3.5.
4. Calculate the difference ($d_i$) between the ranks for each pair of observations.
5. Square each difference ($d_i^2$).
6. Sum the squared differences ($\sum{d_i^2}$).
7. Count the number of data pairs (n).
8. Plug these values into the formula: $ρ = 1 - \frac{ 6 \sum{d_i^2} }{ n(n^2 - 1) }$
9. Calculate the value of ρ.

If there are tied ranks, the simplified formula above is an approximation. The exact method involves calculating Spearman's rho using the Pearson correlation formula on the ranks. However, for most exam purposes, the simplified formula is sufficient, and the question will often specify if an approximation is acceptable or if exact calculation is needed.

Interpretation of Spearman's ρ

The interpretation of Spearman's rho is similar to Pearson's r:

• A value of +1 indicates a perfect positive monotonic relationship.
• A value of -1 indicates a perfect negative monotonic relationship.
• A value of 0 indicates no monotonic relationship.

The strength of the relationship is interpreted using similar guidelines as for Pearson's r, although the thresholds might be slightly adjusted due to the nature of ranked data.

Example of Spearman Correlation

Let's use the same data as the Pearson example, but this time we'll calculate Spearman's rho.

Here, $n=5$.

Using the formula for no tied ranks:

$ρ = 1 - \frac{ 6 \sum{d_i^2} }{ n(n^2 - 1) }$

$ρ = 1 - \frac{ 6 (0) }{ 5(5^2 - 1) }$

$ρ = 1 - \frac{ 0 }{ 5(24) }$

$ρ = 1 - 0 = 1$

In this specific example, the raw scores were perfectly ordered, resulting in a perfect rank correlation of 1. This indicates a perfect positive monotonic relationship between hours studied and exam scores.

Example with Tied Ranks

Consider the following data for two variables X and Y:

$n=5$.

Step 1: Rank X

• Raw X values: 10, 12, 12, 15, 18
• Ranks without ties: 1, 2, 3, 4, 5
• There's a tie for the value 12, which would occupy ranks 2 and 3. The average rank is (2+3)/2 = 2.5.
• Ranked X: 1 (for 10), 2.5 (for 12), 2.5 (for 12), 4 (for 15), 5 (for 18)

Step 2: Rank Y

• Raw Y values: 25, 22, 28, 30, 22
• Ranks without ties: 2, 1, 4, 5, 3
• There's a tie for the value 22, which would occupy ranks 1 and 2. The average rank is (1+2)/2 = 1.5.
• Ranked Y: 2 (for 25), 1.5 (for 22), 4 (for 28), 5 (for 30), 1.5 (for 22)

Step 3: Calculate Differences and Squared Differences

Step 4: Calculate Spearman's ρ

$ρ = 1 - \frac{ 6 \sum{d_i^2} }{ n(n^2 - 1) }$

$ρ = 1 - \frac{ 6 (17.5) }{ 5(5^2 - 1) }$

$ρ = 1 - \frac{ 105 }{ 5(24) }$

$ρ = 1 - \frac{ 105 }{ 120 }$

$ρ = 1 - 0.875 = 0.125$

The Spearman correlation coefficient is 0.125, indicating a very weak positive monotonic relationship between variables X and Y in this dataset.

Assumptions and Limitations of Spearman Correlation

Spearman's rho has fewer assumptions than Pearson's r:

• Monotonic Relationship: It assumes that the relationship between the two variables is monotonic. If the relationship is non-monotonic (e.g., U-shaped), Spearman's rho may not accurately reflect the association.
• Ordinal Data: It is suitable for ordinal data or continuous data that has been ranked.
• Independence: Observations should be independent.

It is less sensitive to outliers than Pearson's r because it uses ranks. However, it may be less powerful than Pearson's r if the assumptions for Pearson's r are met.

Choosing Between Pearson and Spearman

The choice between Pearson's r and Spearman's rho depends on the nature of your data and the relationship you expect to find.

• Pearson's r: Use when you have two continuous variables, the relationship is expected to be linear, and the data is approximately normally distributed.
• Spearman's ρ: Use when you have ordinal data, ranked data, or continuous data where the relationship is monotonic but not necessarily linear, or when the normality assumption is violated. It is also robust to outliers.

A scatterplot of your data is an excellent first step to visually assess the linearity and distribution of your variables, which can help guide your choice of correlation coefficient.

Significance of Correlation

It's important to remember that a high correlation coefficient (either Pearson's or Spearman's) does not necessarily imply causation. Correlation indicates association, but it doesn't explain *why* the variables are associated. There might be a third, unmeasured variable influencing both.

In statistical analysis, we often test the significance of the correlation coefficient to determine if the observed correlation in our sample is likely to exist in the population or if it could have occurred by chance. This typically involves hypothesis testing.

* Null Hypothesis ($H_0$): There is no correlation between the two variables in the population (ρ = 0 or r = 0). * Alternative Hypothesis ($H_a$): There is a correlation between the two variables in the population (ρ ≠ 0 or r ≠ 0).

The p-value obtained from the test indicates the probability of observing such a correlation (or a stronger one) if the null hypothesis were true. A small p-value (typically < 0.05) leads us to reject the null hypothesis and conclude that the correlation is statistically significant.