Inference and Evaluation
This guide explains how to run inference or a complete evaluation using command-line tools. Inference produces probability of TB presence for input images, while evaluation will analyze such output against existing annotations and produce performance figures.
Inference
In inference (or prediction) mode, we input a model, a dataset, a model checkpoint generated during training, and output a json file containing the prediction outputs for every input image.
To run inference, use the sub-command :ref:`predict <mednet.libs.common.cli>`.
Examples
To run inference using a trained Pasa CNN on the Montgomery dataset:
mednet predict -vv pasa montgomery --weight=<path/to/model.ckpt> --output=<results/folder/predictions.json>Replace <path/to/model.ckpt> to a path leading to the pre-trained model.
Evaluation
In evaluation, we input predictions to generate performance summaries that help analysis of a trained model. The generated files are a .pdf containing various plots and a table of metrics for each dataset split. Evaluation is done using the :ref:`evaluate command <mednet.libs.common.cli>` followed by the json file generated during the inference step and a threshold.
Use mednet evaluate --help for more information.
Examples
To run evaluation on predictions generated in the inference step, using an optimal threshold computed from the validation set, do the following:
mednet evaluate -vv --predictions=<path/to/predictions.json> --output-folder=<results/folder> --threshold=validation