Parameter Estimation

Methods find the most likely parameter

\hat\theta

that explain the data

D

and boil down to

\hat{\theta} \in \argmin_\theta \mathcal{L}(\theta)

\Theta \subset R

Risk = bias^2 + variance

statistical experiment be a sample

X_1

… ,

X_n

of i.i.d. random variables in some measurable space Ω, usually Ω ⊆ ℝ

hyperparameter

\alpha

D

is data set

While performing MLE estimation, we update the weights through back propagation to maximize the likelihood of the data, obtaining the optimal point estimation

While performing MAP estimation, we update the weights through back propagation to maximize the posterior probability, obtaining the optimal point estimation

While performing Bayesian inference, we update the weights through back propagation to calculate the posterior probability distribution, obtaining the optimal density estimation

Point Estimations

Parameter Estimation Notion