Refactoring baselines
Hey guys,
Follow the first attempt to replace the baselines.py
script.
I've done it using click. To use it, just type:
bob bio face baselines --help
In here #13 (closed) @mguenther raised some important points that I'm approaching via this MR. And they are the following:
Looking at this, it seems not to be possible to provide generic baseline configuration files, since the configuration depends on the database and the algorithm. For example, we need different preprocessors:
base
foratnt
(which has pre-aligned images) andface-crop-eyes
for other databases, which have annotations. Similarly, we have to specify thegroups
to be['dev']
foratnt
(and some other databases), and['dev', 'eval']
for others likemobio
and alike.
The issue with dev
and eval
groups I'm approaching here bob.bio.base!150 (merged). The FileDatabase API should expose the groups available. In the new baseline script, if the groups
method isn't available, I set it to dev
by default.
About the specific preprocessors, I haven't approached yet. Maybe something like this https://gitlab.idiap.ch/bob/bob.bio.face/blob/8868ffcc2408df1b5d7f20fb4b87e6a9b061cb05/bob/bio/face/script/baselines.py#L205 should be ok, I don't know.
Yet another issue is that -- currently -- the
bin/baselines.py
script also provides the possibility to--evaluate
. With the current evaluation scriptbin/evaluate/py
it is quite difficult to replicate the baseline evaluation, as it automatically check for the result files to be present. However, this could be simplified if we actually redesign./bin/evaluate.py
to do a similar job.
We need to think about this one, but I think wouldn't be a pain with our new evaluation set of scripts implemented in bob.bio.base
Tell me what do you think @theophile.gentilhomme @mguenther @amohammadi
Closes #13 (closed) Closes #30 (closed) Closes #22 (closed)
Cheers