Bayesian Networks Independencies

Independence

1. For events $\alpha,\beta$, $P\models\alpha\perp\beta$ if:
   
   • $P(\alpha,\beta)=P(\alpha)P(\beta)$
   • $P(\alpha|\beta)=P(\alpha)$
   • $P(\beta|\alpha)=P(\beta)$
2. For random variables $X,Y$,$P\models X\perp Y$ if:
   
   • $P(X,Y)=P(X)P(Y)$
   • $P(X|Y)=P(X)$
   • $P(Y|X)=P(Y)$

Conditional Independence

For (sets of) random variables $X,Y,Z$, $P\models(X\perp Y|Z)$ if:
- $P(X,Y|Z)=P(X|Z)P(Y|Z)$
- $P(X|Y,Z)=P(X|Z)$
- $P(Y|X,Z)=P(Y|Z)$
- $P(X,Y|Z)\propto \phi_1(X,Z)\phi_2(Y,Z)$

d-seperated

$X$ and $Y$ are d-seperated in $G$ given $Z$ if there is no active trail in $G$ between $X$ and $Y$ given $Z$, $\text{d-sep}_G(X,Y|Z)$

Factorization $\Rightarrow$ Independence: BNs

• Theorem: If $P$ factorized over $G$, and $\text{d-sep}_G(X,Y|Z)$, then $P$ satisfies $(X\perp Y|Z)$
• Any node is d-seperated from its non-descendants given its parents.
• If $P$ factorizes over $G$, then in $P$, any variable is independent of its non-descendants given its parents.
• I-maps
  
  • d-separation in $G\Rightarrow P$ satisfies corresponding independence statement
    $$I(G)=\{(X,Y|Z): \text{d-sep}_G(X,Y|Z)\}$$
  • Definition: If $P$ satisfies $I(G)$, we say that $G$ is an I-map (independency map) of $P$.
  • Theorem: If $P$ factorized over $G$, then $G$ is an I-map of $P$.

  Independence $\Rightarrow$ Factorization

  • Theorem: If $G$ is an I-map for $P$, then $P$ factorized over $G$.

  Summary

  • Two equivalent views of graph structure
    
    • Factorization: $G$ allows $P$ to be represented
    • I-map: Independencies encoded in $G$ hold in $P$
  • If $P$ factorizes over a graph $G$, we can read from the graph independencies that must hold in (an independency map)