Logistic Regression

Creator

Creator

Created

Created

2023 Mar 16 1:58

Editor

Editor

Edited

Edited

2025 May 20 15:43

Refs

Refs

Sigmoid Function

Gaussian Discriminant Analysis

Data Classification

Transform probability to linear form like
Linear Regression using
Odds ratio

Unlike linear regression, it handles non-linear relationships well while still being mathematically treatable as linear.

It uses a sigmoid function to output values between 0 and 1, maximizing the log-likelihood.

notion image

to make convex cost function

notion image

Logistic Regression Notion

Multi-Class Logistic Regression

h_\theta(x) = g(\theta^Tx) = \frac{1}{1 + e^{-\theta^Tx}}

This leads to

p(u|x;\theta)

described by

Bernoulli Distribution

p(y|x;\theta) = (h_\theta(x))^y(1 - h_\theta(x))^{1 - y}

the

Likelihood function is

l(\theta) = logL(\theta) = \Sigma_{i=1}^ny^{(i)}logh(x^{(i)}) + (1 - y^{(i)})log(1 - h(x^{(i)}))

you can use

Newton–Raphson method or

Stochastic Gradient Descent and prior one achieves faster convergence or

IRLS is much faster

difference between

Linear Regression is that usage of sigmoid function or logistic function

a change of one unit of feature

x_j

changes the odds ratio by a factor

e^{\theta_j}

Logistic Multinomial Regression

Multinomial logistic regression

In statistics, multinomial logistic regression is a classification method that generalizes logistic regression to multiclass problems, i.e. with more than two possible discrete outcomes.[1] That is, it is a model that is used to predict the probabilities of the different possible outcomes of a categorically distributed dependent variable, given a set of independent variables (which may be real-valued, binary-valued, categorical-valued, etc.).

Multinomial logistic regression

https://en.wikipedia.org/wiki/Multinomial_logistic_regression

Backlinks

Discriminative Model

Recommendations

///////