Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • bob.bio.face bob.bio.face
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 22
    • Issues 22
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 5
    • Merge requests 5
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • bobbob
  • bob.bio.facebob.bio.face
  • Merge requests
  • !49

Refactoring baselines

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Tiago de Freitas Pereira requested to merge refactoring-baselines into master May 12, 2018
  • Overview 26
  • Commits 3
  • Pipelines 6
  • Changes 9

Hey guys,

Follow the first attempt to replace the baselines.py script. I've done it using click. To use it, just type:

bob bio face baselines --help

In here #13 (closed) @mguenther raised some important points that I'm approaching via this MR. And they are the following:

Looking at this, it seems not to be possible to provide generic baseline configuration files, since the configuration depends on the database and the algorithm. For example, we need different preprocessors: base for atnt (which has pre-aligned images) and face-crop-eyes for other databases, which have annotations. Similarly, we have to specify the groups to be ['dev'] for atnt (and some other databases), and ['dev', 'eval'] for others like mobio and alike.

The issue with dev and eval groups I'm approaching here bob.bio.base!150 (merged). The FileDatabase API should expose the groups available. In the new baseline script, if the groups method isn't available, I set it to dev by default. About the specific preprocessors, I haven't approached yet. Maybe something like this https://gitlab.idiap.ch/bob/bob.bio.face/blob/8868ffcc2408df1b5d7f20fb4b87e6a9b061cb05/bob/bio/face/script/baselines.py#L205 should be ok, I don't know.

Yet another issue is that -- currently -- the bin/baselines.py script also provides the possibility to --evaluate. With the current evaluation script bin/evaluate/py it is quite difficult to replicate the baseline evaluation, as it automatically check for the result files to be present. However, this could be simplified if we actually redesign ./bin/evaluate.py to do a similar job.

We need to think about this one, but I think wouldn't be a pain with our new evaluation set of scripts implemented in bob.bio.base Tell me what do you think @theophile.gentilhomme @mguenther @amohammadi

Closes #13 (closed) Closes #30 (closed) Closes #22 (closed)

Cheers

Edited May 17, 2018 by Tiago de Freitas Pereira
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: refactoring-baselines