The majority (if not all) of the metrics implemented in bob bio relies on a deveval sets.
Excluding the EPC curve, whose dev and eval sets are necessary for the computation of the curve, this is not a requirement for the rest of the metrics in biometrics (and doesn't make sense).
Furthermore, most of the databases we have don't have the pair dev and eval.
I guess only the ones we created the protocol ourselves have these sets.
How can I use bob bio metric if my dataset has only a dev set?
For instance, if I try to plot a ROC curve I get:
bob bio roc ./scores-dev -o my-roc.pdfError: Invalid value for "scores": The number of provided scores must be > 0 and a multiple of 2 because the following files are required:- 1 development file(s)- 1 evaluation file(s)
This is a severe issue.
For instance, I can't use these scripts for my work (only through a hack)