Variance

The second moment about the mean

Spurious Pattern Comes from randomness of dataset

Variance captures how the random nature of the finite dataset not family of model; It captures how much a model changes if we train on a different training set. i.e. the sensitivity of the model to the randomness in the dataset.

Var(X) = E[(X - E[X])^2] \newline= E[X^2] - 2E[X]E[X] + E[X]^2 \newline = E[X^2] - E[X]^2 \ge 0

Var(X) = Cov(X, X)

Addition rule

V(X \pm Y) = E\left[ (x \pm y)^2 \right] - \left( \mu_{x \pm y} \right)^2 \\ = E\left[ (x \pm y)^2 \right] - \left( \mu_x \pm \mu_y \right)^2 \\ = E\left[ x^2 \pm 2xy + y^2 \right] - \left( \mu_x^2 \pm 2\mu_x \mu_y + \mu_y^2 \right) \\ = E(x^2) \pm 2E(xy) + E(y^2) - \mu_x^2 \mp 2\mu_x \mu_y - \mu_y^2 \\ = E(x^2) - \mu_x^2 \pm 2 \left( E(xy) - \mu_x \mu_y \right) + E(y^2) - \mu_y^2 \\ = V(X) + V(Y) \pm 2\cdot Cov(X, U)

V(X\pm Y) = V(X) + V(Y) \pm2Cov(X, Y)

V(X+Y) = V(X) + V(Y), \text{if X and U are independent}

Variance Usages

Law of total variance

Homoscedasticity

Heteroscedasticity

ANOVA

Generalized variance

Explained Variance

Coefficient of Determination

Variance

The second moment about the mean

Spurious Pattern Comes from randomness of dataset

Addition rule

Recommendations