bob.bio.base merge requestshttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests2020-10-08T11:26:20Zhttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/200Database interface2020-10-08T11:26:20ZTiago de Freitas PereiraDatabase interfaceImplemented a simple filelist database interface for the VanillaBiometrics based on CSVFiles.
The `CSVDatasetDevEval` needs to have the following format:
```text
my_dataset/
my_dataset/my_protocol/
my_datase...Implemented a simple filelist database interface for the VanillaBiometrics based on CSVFiles.
The `CSVDatasetDevEval` needs to have the following format:
```text
my_dataset/
my_dataset/my_protocol/
my_dataset/my_protocol/train.csv
my_dataset/my_protocol/train.csv/dev_enroll.csv
my_dataset/my_protocol/train.csv/dev_probe.csv
my_dataset/my_protocol/train.csv/eval_enroll.csv
my_dataset/my_protocol/train.csv/eval_probe.csv
...
```
where each CSV file needs to have the following format:
```text
PATH,SUBJECT
path_1,subject_1
path_2,subject_2
path_i,subject_j
```
This formart allows the usage of metadata by following the pattern below:
```text
PATH,SUBJECT,METADATA_1,METADATA_2,METADATA_k
path_1,subject_1,A,B,C
path_2,subject_2,A,B,1
path_i,subject_j,2,3,4
```
We can imagine other implementations of this.
For instance, `CSVDatasetCrossValidation` that given a csv file, it splits "on-the-fly" several data for enrolling, probing and training.
Or `CSVDatasetWithEyesAnnotation`, that handles annotations for Face Rec pipelines.
I still need to implement a mechanism that takes `zip` files as input to `CSVDatasetDevEval`.
That way we can ship databases as simple zip files
ping @ydayer @amohammadi
I'll merge this tomorrow.
I need this to support the efforts on `bob.bio.vein`.Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/201Correct the chain loading and click implementation2020-10-13T13:20:31ZAmir MOHAMMADICorrect the chain loading and click implementationThe vanilla biometrics script was not using our click API correctly,
this fixes that. Please refer to bob.extension's docs for more info.The vanilla biometrics script was not using our click API correctly,
this fixes that. Please refer to bob.extension's docs for more info.Bob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/194New documentation bob.bio.base2020-10-14T12:00:53ZTiago de Freitas PereiraNew documentation bob.bio.baseRedoing bob.bio.base documentation
We might rebrand this package to `scikit-biometrics`Redoing bob.bio.base documentation
We might rebrand this package to `scikit-biometrics`Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/203Fix Dask shutting down automatically when calling from a script2020-11-05T13:03:09ZYannick DAYERFix Dask shutting down automatically when calling from a scriptDefers the shutdown task to the user by putting the execution of Vanilla-Biometrics in its own function.
The `bob bio pipelines vanilla-biometrics` command calls this function.Defers the shutdown task to the user by putting the execution of Vanilla-Biometrics in its own function.
The `bob bio pipelines vanilla-biometrics` command calls this function.Bob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/180[dask] Preparing bob.bio.base for dask pipelines2020-11-05T13:56:20ZTiago de Freitas Pereira[dask] Preparing bob.bio.base for dask pipelines**THIS IS A WIP**
Working out how `bob.bio.base` should be with dask pipelines.
Some things to be done in the MR is.
- [x] Remove all traces of `verify.py`. Pipelines needs to be defined with dask. This includes the removal of: `F...**THIS IS A WIP**
Working out how `bob.bio.base` should be with dask pipelines.
Some things to be done in the MR is.
- [x] Remove all traces of `verify.py`. Pipelines needs to be defined with dask. This includes the removal of: `FileSelector`, `Grid` configurations, `./script/grid_search`, `./script/enroll`, `./script/extract`, `./script/preprocess`, `./baselines`
- [x] Move the biometrics pipeline from `bob.pipelines` to here
- [ ] Document the package on: 1-) How to use the resources available (databases, algorithms, preprocessors, etc..), 2-) How to create new databases, 3-) How to create new pipelines (this should be in bob.pipelines)
- [x] Rethink the file list interface
- [ ] Review which dependencies to keep. For instance, `bob.learn.linear`, `bob.learn.em`, `bob.learn.activation`, `bob.math` (consider use scikit-learn).
- [x] Implement missing pipelines (for instance, score normalization/calibration pipeline)
- [x] Database paths patched in `~/.bobrc`
- [ ] Redesign the resource.py as CLI commandsBob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/204Remove stackable preprocessors and extractors2020-11-10T15:21:54ZAmir MOHAMMADIRemove stackable preprocessors and extractorsNow that we have standardized on scikit-learn transformers,
scikit-learn alternatives such as FunctionTransformer, Pipeline,
and FeatureUnion should be usedNow that we have standardized on scikit-learn transformers,
scikit-learn alternatives such as FunctionTransformer, Pipeline,
and FeatureUnion should be usedBob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/202Dask annotators2020-11-10T15:33:49ZAmir MOHAMMADIDask annotatorsBob 9.0.0Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/205Fix annotators kwargs2020-11-11T11:09:14ZYannick DAYERFix annotators kwargsThe kwargs were not passed correctly in the FailSafe annotator.
Removed the shutdown of dask clients in the `annotate` commands.The kwargs were not passed correctly in the FailSafe annotator.
Removed the shutdown of dask clients in the `annotate` commands.Bob 9.0.0Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/206Created decorators2020-11-11T17:31:11ZTiago de Freitas PereiraCreated decoratorsAdded two decorators that checks `mxnet` and `tensorflow`Added two decorators that checks `mxnet` and `tensorflow`Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/207Fixed [typo]2020-11-11T16:04:42ZTiago de Freitas PereiraFixed [typo]https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/208Fixing sphinx warnings2020-11-18T08:16:40ZTiago de Freitas PereiraFixing sphinx warningsIt's a WIP.
I have 34 warnings to be fixed so far.
ping @ydayer @amohammadiIt's a WIP.
I have 34 warnings to be fixed so far.
ping @ydayer @amohammadihttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/209Putting annotator back to the documentation2020-11-18T13:40:26ZTiago de Freitas PereiraPutting annotator back to the documentationhttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/210[annotators][FailSafe] improvements2020-11-19T23:42:16ZAmir MOHAMMADI[annotators][FailSafe] improvementsBob 9.0.0Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/212Improvements on CheckpointWrapper2020-11-23T08:38:04ZTiago de Freitas PereiraImprovements on CheckpointWrapperRelated to https://gitlab.idiap.ch/bob/bob.pipelines/-/merge_requests/48Related to https://gitlab.idiap.ch/bob/bob.pipelines/-/merge_requests/48Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/211[legacy databases] delay loading of annotations2020-11-23T10:03:19ZAmir MOHAMMADI[legacy databases] delay loading of annotationsDepends on bob.pipelines!47Depends on bob.pipelines!47Bob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/213[VanillaBiometrics] Make some dask arguments visible in the CLI commands2020-11-23T10:35:51ZTiago de Freitas Pereira[VanillaBiometrics] Make some dask arguments visible in the CLI commandsSometimes our heuristics that defines `dask.bag.partition_size` and `n_workers` is not enough to cover all range of problems.
In this MR I'm adding the options `--dask-partition-size` and `--dask-n-workers` in the `vanilla-biometrics` CL...Sometimes our heuristics that defines `dask.bag.partition_size` and `n_workers` is not enough to cover all range of problems.
In this MR I'm adding the options `--dask-partition-size` and `--dask-n-workers` in the `vanilla-biometrics` CLI command, so that users can explicitly set, respectively, the dask bag partition size and the number of workers to start an experiment without having to play around with the API.Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/214Optimize scoring if `allow_scoring_with_all_biometric_references == False`2020-11-23T08:43:28ZTiago de Freitas PereiraOptimize scoring if `allow_scoring_with_all_biometric_references == False`..by caching biometric references.
This speeds things up by a lot..by caching biometric references.
This speeds things up by a lotBob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/216[click][VanillaBiometrics] Fixed click options2020-11-23T14:51:12ZTiago de Freitas Pereira[click][VanillaBiometrics] Fixed click optionsCloses #148Closes #148https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/218[test_utils] Do not re-download ATNT database2020-11-23T15:21:39ZAmir MOHAMMADI[test_utils] Do not re-download ATNT databaseif the data is already inside the low-level db package.
Fixes #127if the data is already inside the low-level db package.
Fixes #127Bob 9.0.0Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/219Remove fiddling with bob.pipelines.Samples's internals2020-11-25T09:56:09ZAmir MOHAMMADIRemove fiddling with bob.pipelines.Samples's internalsBob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereira