ERM

Creator

Created

2024 Nov 26 11:26

Editor

Edited

2025 Apr 28 22:50

Refs

\min_{\theta}\frac{1}{n}\sum_{i=1}^n \ell\bigl(f(x_i;\theta),y_i\bigr)+\Omega(\theta)

A generalization of the Maximum Likelihood principle

MLE: replace the log likelihood with any other loss function

l

\mathcal{L}(\theta) = \frac{1}{n} \sum_{i=1}^n \mathcal{l}(y_i, \theta) + \lambda C(\theta)

When a loss function is computationally difficult to minimize, it is often replaced with convex upper bounds

////////