WIP: Implemented two types of metric tables. One for search and one for comparison
I prepared 2 types of plots; one for the comparison protocols and one for the search protocols.
For the comparison, typing this:
$ bob_ijba_collect_results.py --directory <comparison_experiment> --report-type comparison
dumps this
+-----------------+-----------------+-----------------+-----------------+--------------------------+
| CMC% (R=1) | TPIR% (FAR=0.1) | TPIR% (FAR=0.01)|TPIR% (FAR=0.001)| split |
+=================+=================+=================+=================+==========================+
|94.565 |92.069 |66.223 |39.046 |split 0 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.897 |93.086 |69.252 |48.904 |split 1 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.985 |92.683 |67.712 |34.088 |split 2 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|95.938 |93.6 |74.457 |47.301 |split 3 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.47 |91.277 |64.31 |34.322 |split 4 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.053 |93.083 |64.138 |36.286 |split 5 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.431 |94.855 |69.613 |48.91 |split 6 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.863 |91.409 |68.196 |32.497 |split 7 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.572 |91.798 |69.27 |48.539 |split 8 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.565 |92.069 |66.223 |39.046 |split 9 |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
For the search, typing this:
$ bob_ijba_collect_results.py --directory <search_experiment> --report-type search
dumps this:
+-----------------+-----------------+-----------------+--------------------------+
| DIR% (FAR=0.1) | DIR% (FAR=0.01) | DIR% (FAR=0.001)| split |
+=================+=================+=================+==========================+
|51.118 |28.355 |0.0 |split 0 |
+-----------------+-----------------+-----------------+--------------------------+
|52.741 |31.146 |0.0 |split 1 |
+-----------------+-----------------+-----------------+--------------------------+
|53.865 |28.595 |0.0 |split 2 |
+-----------------+-----------------+-----------------+--------------------------+
|49.431 |27.642 |0.0 |split 3 |
+-----------------+-----------------+-----------------+--------------------------+
|43.342 |14.758 |0.0 |split 4 |
+-----------------+-----------------+-----------------+--------------------------+
|56.591 |31.544 |0.0 |split 5 |
+-----------------+-----------------+-----------------+--------------------------+
|46.507 |26.93 |0.0 |split 6 |
+-----------------+-----------------+-----------------+--------------------------+
|51.214 |26.233 |0.0 |split 7 |
+-----------------+-----------------+-----------------+--------------------------+
|51.118 |30.075 |0.0 |split 8 |
+-----------------+-----------------+-----------------+--------------------------+
|51.118 |28.355 |0.0 |split 9 |
+-----------------+-----------------+-----------------+--------------------------+
I set as a WIP, because I still need to solve some stuff that are hard coded (FAR values for instance).
Another not good thing is that I made a dependency with bob.bio.base
(I reused some stuff from bob.bio.base.scripts.collect_results
(bob.bio.base!113 (merged))).
Would be nice to have your input on this if you have the time @mguenther.
Thanks