Categorical covariates

A covariate is a a part of a Linear models that is used as a predictor for the response variable. These can be numeric covariates or categorical covariates.

Categorical covariates are different discrete categories which can be used in the model. In R, this is usually a text field in the data. If you use the as.factor() function on this field, R will know to interpret it as categorical data. This affects the way it gets plotted and treated in a linear model (using the lm() function).

In a linear model, R will assign unique integer numbers to each category (e.g. 0, 1, 2, 3, ...). This number will be used when plotting this covariate in a scatterplot.
If you use the generic plot() function with a factor covariate and another numeric (response) variable, it will make a handy boxplot for comparing the distributions across the different categories.

Generating data in R

By using rnorm() three times

In the lecture notes, they used some R code to simulate sampling data from three distributions using three categories (urban, forest, agriculture). The response variable $y$ was generated with:

"urban" got three records. The fixed effect (constant that is added to each observation) is 1. Then we generate three random numbers from $N \sim (0, 1) + 1$
"Forest" - three records, $N \sim (0, 1) + 5$
"Agriculture" - four records, $N \sim (0, 1) + 3$

Doing this in R:

# categorical covariates
x.raw = c("urban","urban","urban","forest","forest","forest","agriculture","agriculture","agriculture","agriculture")

# sample outcomes
y = c(rep(0,10))
# If the associated x with this y is urban, assign these three numbers
y[x.raw=="urban"] = 1 + rnorm(3,mean = 0,sd=1)
y[x.raw=="forest"] = 5 + rnorm(3,mean = 0,sd=1)
y[x.raw=="agriculture"] = 3 + rnorm(4,mean = 0,sd=1)

Matrix and vector of fixed effects

Another way to do this in a more math-y way is with matrix multiplication. I think we don't really need to worry about this too much.

Create a matrix with three columns (one for each category). In each row, set the column associated with that category to 1 and the others to zero. You'll end up with a matrix like:

X = [\begin{matrix} 1 & 0 & 0 \\ 1 & 0 & 0 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 1 \end{matrix}]

The columns are urban, forest and agriculture. The first three rows are urban, the next three are forest, and the last four are agriculture.

Next we define a matrix of fixed effects (the constant added to each observation): $β = [\begin{matrix} β_{1} \\ β_{2} \\ β_{3} \end{matrix}] = [\begin{matrix} 1 \\ 5 \\ 3 \end{matrix}]$

We do matrix multiplication to multiply these together, which looks like:

\begin{aligned} X β & = [\begin{array}{c} X_{11} & X_{12} & X_{13} \\ ⋮ & ⋱ & ⋮ \\ X_{n 1} & X_{n 2} & X_{n 3} \end{array}] [\begin{array}{c} β_{1} \\ β_{2} \\ β_{3} \end{array}] \\ = [\begin{array}{c} X_{11} β_{1} + X_{12} β_{2} + X_{13} β_{3} \\ ⋮ \\ X_{n 1} β_{1} + X_{n 2} β_{2} + X_{n 3} β_{3} \end{array}] \end{aligned}

That is, mutliplying an $n \times 3$ matrix ( $X$ ) by a $3 \times 1$ vector ( $β$ ), we end up with a $n \times 1$ vector made of the sums of each element. Since in our matrix $X$ we only had zeros and ones, we all the other terms just go to zero. So really what happens is we just assign the associated $β_{i}$ value to each row in a bigger vector:

\begin{aligned} X β & = [\begin{array}{c} X_{11} β_{1} + X_{12} β_{2} + X_{13} β_{3} \\ ⋮ \\ X_{n 1} β_{1} + X_{n 2} β_{2} + X_{n 3} β_{3} \end{array}] \\ = [\begin{array}{c} 1 β_{1} + 0 β_{2} + 0 β_{3} \\ ⋮ \\ 0 β_{1} + 0 β_{2} + 1 β_{3} \end{array}] \\ = [\begin{array}{c} 1 \\ ⋮ \\ 3 \end{array}] \end{aligned}

In R

You can do this in R with:

# Matrix of "covariates"
X = matrix(0, nrow=10, ncol=3)
X[x.raw=="urban",1] = 1
X[x.raw=="forest",2] = 1
X[x.raw=="agriculture",3] = 1


# The beta (=fixed effects) vector
beta = as.vector(c(1,5,3))

# (optional) Take a look at the result of matrix multiplication:
X %*% beta
##      [,1] 
## [1,]    1 
## [2,]    1 
## [3,]    1 
## [4,]    5 
## [5,]    5 
## [6,]    5 
## [7,]    3 
## [8,]    3 
## [9,]    3 
## [10,]   3

# Looks as expected. Now if we use this with rnorm() we can generate our data:
y = X %*% beta + rnorm(10,mean = 0,sd=1)

This gives us similar data to the other way, but it's more efficient if you have more categories.

Building a model

Using the factor() function on our x.raw vector of text strings allows R to read it as a categorical variable and build a linear model:

X.factor = factor(x.raw)
  
model = lm( y ~ 1 + X.factor )
summary(model)

The results of the summary are in the lecture notes.