Probability - Important Formulas

Bayesian

Law of total probability

\sum_{i} P (A | B_{i}) P (B_{i})

Bayes' Theorem

P (B | A) = \frac{P (A | B) P (B)}{P (A)}

Binomial/Poisson

Binomial Distribution

P (k) = (\binom{n}{k}) p^{k} (1 - p)^{n - k}

P (k) = \frac{n!}{k! (n - k)!} p^{k} (1 - p)^{n - k}

In this distribution, the expectation is $n p$ and the variance is $n p (1 - p)$

Poisson Distributions

A special binomial distribution where $n$ is much higher than $p$ and we call $λ = n p$

P (k) = \frac{λ^{k}}{k!} e^{- λ}

Probability of Exponential Decay

This probability density function gives the probability of a particle existing at time $t$ when decaying at rate $α$ .

f (t) = e^{- α t} α

Mean and variance

Expectation of a sum

"The expectation of the sum is the sum of the expectation."

E (ξ + η) = E (ξ) + E (η)

Expectation with a coefficient

With a constant $a$ , or if $a$ and $ξ$ are independent of each other:

E (a ξ) = a E (ξ)

True Variance

V (ξ) = E (ξ^{2}) - [E (ξ)]^{2}

Or

V (ξ) = E [(ξ - E (ξ))^{2}]

Sum of variances

V (ξ + η) = V (ξ) + V (η)

(when $ξ$ and $η$ are independent of each other)

Coefficient with variance

V (a ξ) = a^{2} V (ξ)

(Since variance involves squaring the numbers)

Expectation of sample average

E (\bar{ξ}) = E (ξ)

Variance of the sample average

V (\bar{ξ}) = \frac{1}{n} V (ξ)

This is a random variable about the variance in between our average values in our sample

Standard Error

S E = \sqrt{V (\bar{ξ})} = \sqrt{\frac{1}{n} V (ξ)}

Sample Variance

σ^{2} = \frac{\sum_{i} (ξ_{i} - \bar{ξ})^{2}}{n - 1}

Covariance

C o v (ξ, η) = E [(ξ - E (ξ)) (η - E (η))]

Or alternatively (same thing):

C o v (ξ, η) = E [ξ η] - (E (ξ)) (E (η))

Sample covariance

\frac{\sum_{i = 1}^{n} (ξ_{i} - \bar{ξ}) (η_{i} - \bar{η})}{n - 1}

Correlation coefficient

r = \frac{c o v (ξ, η)}{\sqrt{V (ξ)} \sqrt{V (η)}}

ANOVA (Maths) (F-test)

F statistic given by dividing the sample variance of the average of each group times n:

V (ξ) = n V (\bar{ξ}) = n \frac{\sum_{i = 1}^{k} ({\bar{ξ}}_{i} - \bar{ξ})^{2}}{k - 1}

By the average of the sample variance:

V (ξ) = V (\bar{σ^{2}}) = \frac{1}{k} \sum_{i = 1}^{k} \frac{\sum_{j = 1}^{n} (ξ_{i j} - {\bar{ξ}}_{i})^{2}}{n - 1}

Simplified:

F = \frac{n V (\bar{ξ})}{V (\bar{σ^{2}})}

A value close to 1 means the variance calculated by both methods is similar and all groups are part of the same population.

Continuous Distributions

Given a probability density function $f (x)$ , the probability of a value being between $a$ and $b$ is given by the integral:

P (a < ξ < b) = \int_{a}^{b} f (x) d x

Z-test

Used when the true expectation and variance are known.

z = \frac{\bar{ξ} - m}{\sqrt{\frac{V}{n}}} \approx N (0, 1)

t-test

Used when the true expectation but not the variance are known. Note that $σ^{2}$ here refers to the true variance not the sample variance for some reason.

t = \frac{\bar{ξ} - m}{\sqrt{\frac{σ^{2}}{n}}}

Two-sample t-test

Is there a significant difference between the two groups?

t = \frac{{\bar{ξ}}_{1} - {\bar{ξ}}_{2}}{\sqrt{\frac{2 σ^{2}}{n}}}