Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • bob.db.ijba bob.db.ijba
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • bobbob
  • bob.db.ijbabob.db.ijba
  • Merge requests
  • !9

WIP: Implemented two types of metric tables. One for search and one for comparison

  • Review changes

  • Download
  • Email patches
  • Plain diff
Closed Tiago de Freitas Pereira requested to merge issue-7 into master Nov 08, 2017
  • Overview 12
  • Commits 3
  • Pipelines 2
  • Changes 3

I prepared 2 types of plots; one for the comparison protocols and one for the search protocols.

For the comparison, typing this:

$ bob_ijba_collect_results.py --directory <comparison_experiment> --report-type comparison

dumps this

+-----------------+-----------------+-----------------+-----------------+--------------------------+
|    CMC% (R=1)   | TPIR% (FAR=0.1) | TPIR% (FAR=0.01)|TPIR% (FAR=0.001)| split                    |
+=================+=================+=================+=================+==========================+
|94.565           |92.069           |66.223           |39.046           |split 0                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.897           |93.086           |69.252           |48.904           |split 1                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.985           |92.683           |67.712           |34.088           |split 2                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|95.938           |93.6             |74.457           |47.301           |split 3                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.47            |91.277           |64.31            |34.322           |split 4                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.053           |93.083           |64.138           |36.286           |split 5                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.431           |94.855           |69.613           |48.91            |split 6                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|93.863           |91.409           |68.196           |32.497           |split 7                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.572           |91.798           |69.27            |48.539           |split 8                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+
|94.565           |92.069           |66.223           |39.046           |split 9                   |
+-----------------+-----------------+-----------------+-----------------+--------------------------+

For the search, typing this:

$ bob_ijba_collect_results.py --directory <search_experiment> --report-type search

dumps this:

+-----------------+-----------------+-----------------+--------------------------+
| DIR% (FAR=0.1)  | DIR% (FAR=0.01) | DIR% (FAR=0.001)| split                    |
+=================+=================+=================+==========================+
|51.118           |28.355           |0.0              |split 0                   |
+-----------------+-----------------+-----------------+--------------------------+
|52.741           |31.146           |0.0              |split 1                   |
+-----------------+-----------------+-----------------+--------------------------+
|53.865           |28.595           |0.0              |split 2                   |
+-----------------+-----------------+-----------------+--------------------------+
|49.431           |27.642           |0.0              |split 3                   |
+-----------------+-----------------+-----------------+--------------------------+
|43.342           |14.758           |0.0              |split 4                   |
+-----------------+-----------------+-----------------+--------------------------+
|56.591           |31.544           |0.0              |split 5                   |
+-----------------+-----------------+-----------------+--------------------------+
|46.507           |26.93            |0.0              |split 6                   |
+-----------------+-----------------+-----------------+--------------------------+
|51.214           |26.233           |0.0              |split 7                   |
+-----------------+-----------------+-----------------+--------------------------+
|51.118           |30.075           |0.0              |split 8                   |
+-----------------+-----------------+-----------------+--------------------------+
|51.118           |28.355           |0.0              |split 9                   |
+-----------------+-----------------+-----------------+--------------------------+

I set as a WIP, because I still need to solve some stuff that are hard coded (FAR values for instance). Another not good thing is that I made a dependency with bob.bio.base (I reused some stuff from bob.bio.base.scripts.collect_results (bob.bio.base!113 (merged))).

Would be nice to have your input on this if you have the time @mguenther.

Thanks

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: issue-7