ANOVA (Stats)

We also studied this from a different angle in the Math Methods in Biology course: ANOVA (Maths)

Variance decomposition

Last week, we compared two populations using a t-test. If we want to compare more than two populations, we can't use a t-test.
One tool to do this is Analysis of Variance (ANOVA)

Law of Total Variance

\underset{total variance}{\underset{⏟}{var [Y]}} = \underset{within groups variance}{\underset{⏟}{E [var [Y | X]]}} + \underset{between groups variance}{\underset{⏟}{var [E [Y | X]]}} .

Where:

$Y$ is a vector of observations, e.g. trait values
$X$ is a vector of indicators telling which group each individual measurement comes from.
- Like how we used X = 0 or 1 in last week's example comparing two populations with linear regression
This only holds if the residual variance is independent from the variance in covariates - if there is correlation there are some problems, but we won't worry about this for now

Proof: Wikipedia

Since this relation is only true if both come from the same population, we can determine how much the left side differs from the right side and this can tell us how likely it is that our two samples came from the same population.

Example

We will return from our example from last week (comparing whitefish populations in the Baltic Sea).

Population 1: true mean $μ_{1} = 20$ and $σ$ =10,
- Sample $n = 15$
Population 2: true mean $μ_{2} = 30$ and $σ = 10$ ,
- Sample $n = 15$
  In R, we can create a vector of observed observations and covariates just like we did last week (Comparing two populations through linear regression)
  !lecture_notes_week2, p.2

The following math term is a way of saying that the expectation of $Y_{i}$ depends on which group it is in.

E [Y_{i} | X_{i}] = {\begin{cases} μ_{1} if X_{i} = 0 (Population 1) \\ μ_{2} if X_{i} = 1 (Population 2) . \end{cases}

Since for this example we are pretending we don't know what the true population means are, we can calculate the sample means from both groups and assign these to a vector of the same length:

mu = c() mu[X==0] = mean(Y[X==0]) # this is the same as mean(y) 
mu[X==1] = mean(Y[X==1]) # this is the same as mean(z)

Components

Total variance

Since both samples are the same size ( $n$ ), the size of the entire vector is $2 n$ . So the formula for the total variance is:

var [Y] = \frac{1}{2 n} \sum_{i = 1}^{2 n} (Y_{i} - \bar{Y})^{2}

Within Groups Variance

This is equal to the average of the variances within each group. So we add the variance from population 1 and population 2, then divide by two (the number of groups):

E [var [y | x]] = \frac{1}{2} (var [Y_{i} | X_{i} = 0]] + var [Y_{i} | X_{i} = 1]])

Now, doing some algebra, given the formulas for true variance from the maths class, we can simplify this to:

E [var [y | x]] = \frac{1}{2 n} \sum_{i = 1}^{2 n} (Y_{i} - E [Y_{i} | X_{i}])^{2}

(remember that $E [Y_{i} | X_{i}]$ can be either $μ_{1}$ or $μ_{2}$ depending on $X_{i}$ ). In R, we simply do var(Y - mu). (where mu is our vector of sample means)

Between Groups Variance

The mean of the group means is $\bar{μ} = \frac{1}{2} (μ_{1} + μ_{2})$ . We use this for the between groups variance calculation:

var [E [y | x]] = \frac{1}{2} ((μ_{1} - \bar{μ})^{2} + (μ_{2} - \bar{μ})^{2})

This can be simplified to:

var [E [y | x]] = var [μ - \bar{μ}]

And in R, simply var(mu-mean(Y))

Proportion of explained variance

Now that we have decomposed the variance into components, we can analyze the proportion of variance explained by the different groups:

variance explained = \frac{between group variance}{total variance}

= \frac{var [E [Y [X]]}{var [Y]}

Note that with some algebra you can show that this is actually the same as the multiple r-squared. We can show this in R by comparing doing this calculation manually with the R-squared value from a linear model.

F-statistic

F-statistic
Ok, this is how this ANOVA analysis works with two populations. But the point of this class was to examine more than two. We use the F-statistic to do this, using the Residual sum of squares.

The F-statistic is the quotient of the between groups variance with the within groups variance:

F = \frac{between group variance}{within groups variance}

In statistical software, the F test used is a bit different and looks more like:

F = \frac{\overset{bg sum of squares}{\overset{⏞}{\sum_{i = 1}^{k n} (μ_{i} - \bar{Y})^{2}}} / (k - 1)}{\underset{residual sum of squares}{\underset{⏟}{\sum_{i = 1}^{k n} (Y_{i} - μ_{i})^{2}}} / k (n - 1)} = \frac{k (n - 1) \times [between groups variance]}{(k - 1) \times [within group variance]}

Where $k$ representes the number of groups
This term with $k$ is the degrees of freedom

$k - 1$ is the d.f. between groups
$k (n - 1)$ is the d.f. in the Residual sum of squares (within groups)