Scenarios

• Completely observed GMs
  • directed.
  • undirected.
• Partially or Unobserved GMs
  • directed.
  • undirected.

Estimation principles

• MLE.
• Bayesian estimation.
• Maximum conditional likelihood.
• Maximum “Margin”.
• Maximum entropy.

Parameter Learning

• Assume $\mathcal{G}$ is known and fixed,

  • from expert design.
  • from an intermediate outcome of iterative structure learning.

• $D = \{x_1, \dots ,x_N\}$: $N$ independent, identically distributed (iid) training cases.

  • $x_n = (x_{n, 1}, \dots, x_{n, M})$ is a vector of $M$ values
    • the model is observed, every $x_n$ is known
    • or, paritially observed.

• Goal: Estimate from $D$.

Consider learning params for a BN given the structure and is completely observable. The likelihood of the data $$ I(\theta | D) = \log p(D | \theta) = \log \left( \prod_{i} \prod_{j} p(x_{i,j} | \theta) \right) = \sum_{i} \sum_{j} \log p(x_{i,j} | \theta) $$