Guys,

The majority (if not all) of the metrics implemented in bob bio relies on a dev eval sets.

Excluding the EPC curve, whose dev and eval sets are necessary for the computation of the curve, this is not a requirement for the rest of the metrics in biometrics (and doesn't make sense).

Furthermore, most of the databases we have don't have the pair dev and eval. I guess only the ones we created the protocol ourselves have these sets.

How can I use bob bio metric if my dataset has only a dev set?

For instance, if I try to plot a ROC curve I get:

bob bio roc ./scores-dev -o my-roc.pdf

Error: Invalid value for "scores": The number of provided scores must be > 0 and a multiple of 2 because the following files are required:
- 1 development file(s)
- 1 evaluation file(s)

This is a severe issue. For instance, I can't use these scripts for my work (only through a hack)

ping @theophile.gentilhomme and @amohammadi