UCL Probabilistic Modeling Midterm

Mock exam

field of study that gives computers the ability to learn without being explicitly programmed

ML is Function approximation that learns algorithm based on data

The use of probability distributions to represent models, perform inference, and quantify uncertainty.

Definition

”deterministic” produces same output from same initial conditions, without randomness or variation
Frequentist interprets probability as the long-run frequency of an event occurring in repeated experiments in identical settings, admitting the existence of variation, variability and randomness in the world.
”Bayesian” interprets probability as a degree of belief, which is updated as more evidence. They believe that is not possible to run multiple independent trials, essentially treating everything as random.

Machine learning algorithms take data as input (and possibly some parameters), and returns a prediction for a particular task.

Definition

Loss - A function that quantifies how a predictor is a good approximation of Y
Risk - The risk of a predictor is the expected loss when applied to the entire data distribution.
Empirical Risk - average loss calculated on a finite sample of data from the training set.

What functional object are we aiming to build in prediction? Posterior

book1.pdf

90152.8KB

5V

Volume - Data at Rest

Velocity - Data in Motion

Variety - Data in Many Forms

Veracity - Data in Doubt

Value

Tips

이산확률 나오면 모든 경우 나눠서 e, v

Memo

Data Classification vs
Regression

Online learning: observations are revealed over time.

Semi-supervised Learning,
Self-Supervised Learning

Representation matters since different representation suits for statistic model

Definition(s) of convexity and why it matters in machine learning Global Optimum Guarantee

What is generalization? To predict un-seen data

Can a function be convex and concave? Justify. Linear

Bayes Theorem

Probability theory,
Cox's theorem

Marginalization
Law of total variance
Law of total expectation’

Joint Probability
Covariance
Correlation

At a fundamental level, assume there is a sample space consisting of all possible events or outcomes: we can use it to describe our uncertainty around which event will occur.