Baseline-style python configuration

Created by: anjos

I'm trying to revive/port an old package (was called bob.fingervein), based on the old facereclib.

One of the tasks I have is to try and revive baselines on a paper. Though this package provides a somewhat easy way to re-run baselines, I think we could slightly improve it by introducing a "configuration script"-style way to run baselines.

Here is what could be the logic behind it:

The user defines a python-syntax "config" file, which matches the flexibility of verify.py in terms of options. It could be something like this:

import bob.db.vera from bob.bio.base.database import DatabaseBob

database = DatabaseBob(database = bob.db.vera.Database(...)) protocol = 'NOM' #select here the protocol you want to run against groups = ('dev',) #select here the subsets inside the protocol to generate scores for

from bob.bio.vein.preprocessors import FingerCrop

this the preprocessing step for this baseline

preprocessor = FingerCrop() skip_preprocessing = False # set the following flag to True to skip the preprocessing step

more options for 'extrator' and 'algorithm' will follow

setup the grid processing options

from bob.bio.base.grid import Grid grid = Grid(...)

2. The command line then, *may* become as simple as:

```sh
$ ./bin/verify.py -vv config.py

Options on the command line override those in the script. So, the following should work (e.g., to override the grid processing options):

$ ./bin/verify.py -vv config.py --parallel=4


4. Optionally, concatenating multiple configuration files, make it override previously set options:

```sh
$ ./bin/verify.py -vv config1.py config2.py

Config files may be also hooked on the resources/python-entry-point system:

$ ./bin/verify.py vera-baseline1-btas2016


6. The changes are done in a backward-compatible way, so current use of `verify.py` is not affected

This would allow us to manage single files and even exchange baselines with people with full configurations and comments. All lives inside a single, while flexible file and provides a full pythonic way to describe a baseline, that is easy to run with a simple command line.

People around the office here claimed that is possible if we do something like in this test:

https://github.com/bioidiap/bob.bio.base/blob/master/bob/bio/base/test/test_scripts.py#L106

While this would be an alternative, this is not a clean option. Imports would have to be handled in special way and the final script would look very counter-intuitive.

Am I overlooking something?

*Created by: anjos*

I'm trying to revive/port an old package (was called `bob.fingervein`), based on the old facereclib.

Here is what could be the logic behind it:

1. The user defines a python-syntax "config" file, which matches the flexibility of `verify.py` in terms of options. It could be something like this:

```python
import bob.db.vera
from bob.bio.base.database import DatabaseBob

database = DatabaseBob(database = bob.db.vera.Database(...))
protocol = 'NOM' #select here the protocol you want to run against
groups = ('dev',) #select here the subsets inside the protocol to generate scores for

from bob.bio.vein.preprocessors import FingerCrop

# this the preprocessing step for this baseline
preprocessor = FingerCrop()
skip_preprocessing = False # set the following flag to True to skip the preprocessing step

# more options for 'extrator' and 'algorithm' will follow

# setup the grid processing options
from bob.bio.base.grid import Grid
grid = Grid(...)
   ```
2. The command line then, *may* become as simple as:

```sh
$ ./bin/verify.py -vv config.py
   ```

3. Options on the command line **override** those in the script. So, the following should work (e.g., to override the grid processing options):

```sh
$ ./bin/verify.py -vv config.py --parallel=4
   ```

4. Optionally, concatenating multiple configuration files, make it override previously set options:

```sh
$ ./bin/verify.py -vv config1.py config2.py
   ```

5. Config files may be also hooked on the resources/python-entry-point system:

```
$ ./bin/verify.py vera-baseline1-btas2016
   ```

6. The changes are done in a backward-compatible way, so current use of `verify.py` is not affected

This would allow us to manage single files and even exchange baselines with people with full configurations and comments. All lives inside a single, while flexible file and provides a full pythonic way to describe a baseline, that is easy to run with a simple command line.

People around the office here claimed that is possible if we do something like in this test:

https://github.com/bioidiap/bob.bio.base/blob/master/bob/bio/base/test/test_scripts.py#L106

While this would be an alternative, this is not a clean option. Imports would have to be handled in special way and the final script would look very counter-intuitive.

Am I overlooking something?