I prepared 2 types of plots; one for the comparison protocols and one for the search protocols.

For the comparison, typing this:

$ bob_ijba_collect_results.py --directory <comparison_experiment> --report-type comparison

dumps this

+-----------------+-----------------+-----------------+-----------------+--------------------------+
|    CMC% (R=1)   | TPIR% (FAR=0.1) | TPIR% (FAR=0.01)|TPIR% (FAR=0.001)| split                    |
+=================+=================+=================+=================+==========================+
|94.565           |92.069           |66.223           |39.046           |split 0                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.897           |93.086           |69.252           |48.904           |split 1                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.985           |92.683           |67.712           |34.088           |split 2                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|95.938           |93.6             |74.457           |47.301           |split 3                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.47            |91.277           |64.31            |34.322           |split 4                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.053           |93.083           |64.138           |36.286           |split 5                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.431           |94.855           |69.613           |48.91            |split 6                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.863           |91.409           |68.196           |32.497           |split 7                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.572           |91.798           |69.27            |48.539           |split 8                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.565           |92.069           |66.223           |39.046           |split 9                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+

For the search, typing this:

$ bob_ijba_collect_results.py --directory <search_experiment> --report-type search

dumps this:

+-----------------+-----------------+-----------------+--------------------------+
| DIR% (FAR=0.1)  | DIR% (FAR=0.01) | DIR% (FAR=0.001)| split                    |
+=================+=================+=================+==========================+
|51.118           |28.355           |0.0              |split 0                   |
+-----------------+-----------------+-----------------+--------------------------+
|52.741           |31.146           |0.0              |split 1                   |
+-----------------+-----------------+-----------------+--------------------------+
|53.865           |28.595           |0.0              |split 2                   |
+-----------------+-----------------+-----------------+--------------------------+
|49.431           |27.642           |0.0              |split 3                   |
+-----------------+-----------------+-----------------+--------------------------+
|43.342           |14.758           |0.0              |split 4                   |
+-----------------+-----------------+-----------------+--------------------------+
|56.591           |31.544           |0.0              |split 5                   |
+-----------------+-----------------+-----------------+--------------------------+
|46.507           |26.93            |0.0              |split 6                   |
+-----------------+-----------------+-----------------+--------------------------+
|51.214           |26.233           |0.0              |split 7                   |
+-----------------+-----------------+-----------------+--------------------------+
|51.118           |30.075           |0.0              |split 8                   |
+-----------------+-----------------+-----------------+--------------------------+
|51.118           |28.355           |0.0              |split 9                   |
+-----------------+-----------------+-----------------+--------------------------+

I set as a WIP, because I still need to solve some stuff that are hard coded (FAR values for instance). Another not good thing is that I made a dependency with bob.bio.base (I reused some stuff from bob.bio.base.scripts.collect_results (bob.bio.base!113 (merged))).

Would be nice to have your input on this if you have the time @mguenther.

Thanks