Logistic Regression

Creator
Creator
Seonglae Cho
Created
Created
2023 Mar 16 1:58
Editor
Edited
Edited
2025 May 20 15:43

Transform probability to linear form like
Linear Regression
using
Odds ratio

Unlike linear regression, it handles non-linear relationships well while still being mathematically treatable as linear.
It uses a sigmoid function to output values between 0 and 1, maximizing the log-likelihood.
notion image
to make convex cost function
notion image
Logistic Regression Notion
hθ(x)=g(θTx)=11+eθTxh_\theta(x) = g(\theta^Tx) = \frac{1}{1 + e^{-\theta^Tx}}
This leads to p(ux;θ)p(u|x;\theta) described by
Bernoulli Distribution
p(yx;θ)=(hθ(x))y(1hθ(x))1yp(y|x;\theta) = (h_\theta(x))^y(1 - h_\theta(x))^{1 - y}l(θ)=logL(θ)=Σi=1ny(i)logh(x(i))+(1y(i))log(1h(x(i)))l(\theta) = logL(\theta) = \Sigma_{i=1}^ny^{(i)}logh(x^{(i)}) + (1 - y^{(i)})log(1 - h(x^{(i)}))
you can use
Newton–Raphson method
or
Stochastic Gradient Descent
and prior one achieves faster convergence or
IRLS
is much faster
difference between
Linear Regression
is that usage of sigmoid function or logistic function
a change of one unit of feature xjx_jchanges the odds ratio by a factor eθje^{\theta_j}
 

Logistic Multinomial Regression

 

Recommendations