Create the set of observations

We first have to import the OpenBayes package and some other packages (such as copy) that we will need

from OpenBayes import learning, bayesnet
from copy import deepcopy
from time import time

Then we will generate our observations by sampling the Water-Sprinkler network. Please note that this is a tutorial and that in real life you don't generate your observations this way. Usually the set of observations is already available (from an experiment for example). We then delete some data to work with incomplete observations. Put '?' for unknown data.

# first create a bayesian network
from WaterSprinkler import *

N = 2000
# sample the network N times
cases = G.Sample(N)    # cases = [{'c':0,'s':1,'r':0,'w':1},{...},...]
# delete some observations
for i in range(500):
    case = cases[3*i]
    rand = random.sample(['c','s','r','w'],1)[0]
    case[rand] = '?' 
for i in range(50):
    case = cases[3*i]
    rand = random.sample(['c','s','r','w'],1)[0]
    case[rand] = '?'

Now cases contains 2000 dictionaries, each one of them containing one sampled value (or '?' if the value is unknown) for each node in the Water-Sprinkler BN.

If the data is stored in an excell file, you can use the ReadFile function (see previous tutorial).

Create a BN with no edges

We then create a new BN with the same nodes, but with no edges.

# Create a new BNet with no edges
G2 = bayesnet.BNet('Water Sprinkler Bayesian Network2')
c,s,r,w = [G2.add_v(bayesnet.BVertex(nm,True,2)) for nm in 'c s r w'.split()]
G2.InitDistributions()

SEM Algorithm

# Learn the structure
struct_engine = learning.SEMLearningEngine(G2)
struct_engine.SEMLearning(cases)
print 'learned structure: ', struct_engine.BNet
print 'total bic score: ', struct_engine.GlobalBICScore(N, cases)

StructLearning uses the SEM algorithm to learn the structure: it begins with an initial structure, then adds, deletes and reverses all possible edges and keeps the transformation that has the highest BIC score, until no better structure is possible. Each time it checks if a structure is better, it estimates the parameters with the EM algorithm. That method provides a best local structure, but not the best structure. It is therefore recommended to run the algorithm several times with different initial structures.

The SEM algorithm is extremely slow. That is due to the fact that at each check (each time it deletes, adds, or reverses an edge) it reestimates the parameters with the EM algorithm.

SEM Algorithm approximation

# Learn the structure
struct_engine = learning.SEMLearningEngine(G2)
struct_engine.SEMLearningApprox(cases)
print 'learned structure: ', struct_engine.BNet
print 'total bic score: ', struct_engine.GlobalBICScore(N, cases)

The approximation uses the same concept as the "loopy belief algorithm". Instead of using the EM algorithm at each check (each time it deletes, adds, or reverses an edge), it estimates the parameters by looking only at the parents of the nodes, without doing an inference on all the BN. The EM algorithm is then only used one time: at the end of each iteration to correct the parameters for the next iteration.

This method is way faster than the classic SEM algorithm. For the tests that I did, both method gave the same results, however, I can't guarantee the results given by the approximation. The best way to act for learning from incomplete data is to use the result obtained by the approximation as initial graph for the classic SEM algorithm.

The results can easily be saved in a txt file (see previous tutorial).