Test Net Using Cases

The purpose of this test is to grade a Bayes net using a set of real cases to see how well the predictions or diagnosis of the net match the actual cases.  It is not for decision nets.

First select the nodes you do not wish the net to know the value of during its inference.  For example, if the net is for medical diagnosis, you might select the disease node and nodes representing other unobservable internal states.  These nodes are called unobserved nodes.

Then choose Cases Test with Cases.  You will be asked which case file to use, and after you choose one, Netica will start processing.  The Messages window will come to the front and display the fraction of cases processed so far.  Hold down ctrl+shift+left button at the same time if you want to stop processing cases and print the results obtained so far.

Netica will pass through the case file, processing cases one-by-one.  For each case, Netica reads in the case, except for any findings for the unobserved nodes.  It then does belief updating to generate beliefs for each of the unobserved nodes.  It goes back and checks the true value for those nodes as supplied by the case file (if they are supplied for that case), and compares them with the beliefs it generated. It accumulates all the comparisons into summary statistics.

When Netica is done, it will print a report for each of the unobserved nodes.  These reports include a confusion matrix, error rate, calibration table, quadratic (Brier) score, logarithmic loss score, spherical payoff score, surprise indexes, and test sensitivity.  For binary nodes it also reports test specificity, predictive value and predictive value negative.  For an easy way to produce receiver operating characteristic (ROC) plots, see Quality of Test topic.

For full documentation on this function, and the reports generated, see the Test Net with Cases Chapter in Special Topics.