Bn Learner?

[kenneth-lee-ch]

Hi, if I have some discrete dataset, I want to impose a DAG structure on this dataset and and refill the potential based on the observed data and be able to derive the joint distribution according to factorization so that I can do a posterior update with some prior...I remember bnlearner is something relevant, but I am not sure. Would you please provide some pointers?

[phwuil]

Hi @kenneth-lee-ch , sorry or the delay :-)

When you know the structure and just want to "learn" the parameters from a database, the functions is learnParameters, you can choose a priori (for instance, a smoothing).

https://pyagrum.readthedocs.io/en/1.17.2/BNLearning.html#pyAgrum.BNLearner.learnParameters

https://pyagrum.readthedocs.io/en/1.17.2/notebooks/12-Examples_KaggleTitanic.html#Making-a-BN-without-learning-data-(structureFromExpert4)

[kenneth-lee-ch]

If I just want to count the frequency to fill the CPT, is learnParameters() the right option?

[kenneth-lee-ch]

I did this instead and the error is gone:

learner = gum.BNLearner(df)
learner.useSmoothingPrior()
 learned_bn = learner.learnParameters(bn.dag())

[phwuil]

Hi @kenneth-lee-ch ,
The problem seems to be that the BN you use as a template and container for the frequencies you want to calculate from the database is not correcty defined:

[pyAgrum] Unknown label found in database: The translation of String "3" could not be found for variable '1:Range([0,1])'

When creating the variable '1', you specify it as binary (by default I guess), but this column can contain some '3's ...

In order to correctly create your template, you can use a pyAgrum.Discretizer which parses the database and infers the node types:

[kenneth-lee-ch]

@phwuil I tried your code with a randomly generated data test.csv
, but I still get the error.

import pyAgrum as gum
import pandas as pd
df = pd.read_csv('test.csv', index_col=0)
edges = [(0, 1), (2, 0), (2, 1), (2, 8), (3, 2), (3, 4), (3, 6), (3, 8), (6, 7), (6, 8), (5, 6), (9, 5), (10, 0)]
from pyAgrum.lib.discretizer import Discretizer
discretizer = Discretizer()
bn = discretizer.discretizedTemplate(df)
for edge in edges:
    u, v, = edge
    bn.addArc(str(u), str(v))

learner = gum.BNLearner(df, bn)
learner.useNoPrior()
learner.fitParameters(bn)

Error message:

DatabaseError: [pyAgrum] Database error: The conditioning set <10=1, 2=1> for target node 0 never appears in the database. Please consider using priors such as smoothing.

[kenneth-lee-ch]

The dataset only has a single row with the column '10' being 1, so I don't follow what the fitParaemter() is looking up for the combination <10=1, 2=1>.

[kenneth-lee-ch]

Another issue I have is the memory usage...as I repeatedly call BNLearner, it seems to stack up a lot of hard disk space for some reason. Specifcally, I mean this line

learner = gum.BNLearner(joint_df, bn)

Would you mind to explain why even if I constantly replace the BNLearner with a different BN?