Graphical Model

Creator
Creator
Seonglae Cho
Created
Created
2024 Oct 22 23:56
Editor
Edited
Edited
2025 Apr 28 13:22

Probabilistic graphical model (PGM), Structured Probabilistic Model

Joint Distribution
that uses a graph structure to encode conditional independence assumptions. The nodes are random variables, if they have an edge between them they are directly dependent, if they do not they are
Conditionally Independent
.
In a directed graph, if a node sends an arrow, it goes into the conditional part, and if it receives an arrow, it goes into the marginal part. Multiplication is done as many times as the number of nodes. When ABCA\rightarrow B \rightarrow C, AA and BB are predecessors of CC, while AA is not a parent of CC (only B is). In other words, CABC\perp A|B.
YiYpred(i)\pa(i)Ypa(i)Y_i\perp Y_{pred(i) \backslash pa(i)}|Y_{pa(i)}

From factorized probability to
Directed Acyclic Graph

Independent random variables are nodes without any previous input. Random variables which are conditional on others will have an arrow from those conditional variables to themselves.
p(Y1:NG)=i=1NGp(YiYpa(i))p(Y_{1:N_G}) = \prod_{i=1}^{N_G} p(Y_i \mid Y_{\text{pa}(i)})
where NGN_G denotes the number of nodes in the graph. For example when ABCA\rightarrow B\rightarrow C, P(A,B,C)=P(BA)P(CB)P(A,B,C) = P(B|A) P(C|B).
Without
Collider
, every directional graphical model is identical. Collider is a variable that receives multiple arrows. Fixing collider variable make variables dependent between parents.
notion image
p(x3x1,x2)p(x1)p(x2)=p(x1,x2,x3)p(x1x3)p(x2x3)p(x3)=p(x2x3)p(x3,x1)=p(x1x3)p(x3,x2)=p(x2x3)p(x3x1)p(x1)=p(x2x3)p(x3,x1)=p(x2x3)p(x3x1)p(x1)=p(x1x3)p(x3,x2)=p(x1,x2,x3)\begin{aligned} p(x_3 \mid x_1, x_2)\, p(x_1)\, p(x_2) &= p(x_1, x_2, x_3) \\ \ne p(x_1 \mid x_3)\, p(x_2 \mid x_3)\, p(x_3) &= p(x_2 \mid x_3)\, p(x_3, x_1) = p(x_1 \mid x_3)\, p(x_3, x_2) \\ =p(x_2 \mid x_3)\, p(x_3 \mid x_1)\, p(x_1) &= p(x_2 \mid x_3)\, p(x_3, x_1) \\ =p(x_2 \mid x_3)\, p(x_3 \mid x_1)\, p(x_1) &= p(x_1 \mid x_3)\, p(x_3, x_2) \\ = p(x_1, x_2, x_3) \end{aligned}

From DAG to joint probability distribution

All of the variables have a joint probability, the nodes which do not have any arrows pointing to them are independent, the nodes which have arrows pointing to them are dependent of the nodes where the arrows originate.

Un-directed graphs

When the graph is un-directed, the probabilistic distribution is called a
Markov Random Field
. Un-directed graph is more general and indicates less information than directed graphs.
p(X1,,XN)=1ZcCϕc(Xc)p(X_1, \ldots, X_N) = \frac{1}{Z} \prod_{c \in C} \phi_c(X_c)
where CC denotes the set of
Clique
s, and each factor ϕC\phi_C is a non-negative function over the clique cc. Note that ZZ is called the partition function. For example when ABCA-B-C, P(A,B,C)=1ZϕAB(A,B)×ϕBC(B,C)P(A,B,C) = \frac{1}{Z}\phi_{AB}(A,B) \times \phi_{BC}(B,C).
Graphical Model Notion
 
 
 
 
 
Note that the direction of arrows indicates causality, not the direction of inference
 
 
 

Recommendations