bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2023-11-03T09:59:05Zhttps://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/29CI: Database interfaces return empty sample list2023-11-03T09:59:05ZYannick DAYERCI: Database interfaces return empty sample list[Sometimes](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/364632) one of the CI jobs fails because a database returned an empty list when retrieving samples.
Has Mainly been seen on the `macos-arm` jobs, but always on one of both jobs...[Sometimes](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/364632) one of the CI jobs fails because a database returned an empty list when retrieving samples.
Has Mainly been seen on the `macos-arm` jobs, but always on one of both jobs (`3.9,macos-arm` or `3.10,macos-arm`).
Re-running the failed job fixes the issue.
It could be an issue with having two jobs running on the same machine. If both share the same `data` directory: if the first job finishes and cleans the database definition file, the second one has no file to read and returns an empty sample list.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/195download_file extracts the archive file every time it is called.2023-10-31T13:42:42ZYannick DAYERdownload_file extracts the archive file every time it is called.When the `extract` flag is set in [`download_file`](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/v8.0.0/src/bob/bio/base/database/utils.py?ref_type=tags#L474), the archive file is extracted at every call, even if the first call that d...When the `extract` flag is set in [`download_file`](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/v8.0.0/src/bob/bio/base/database/utils.py?ref_type=tags#L474), the archive file is extracted at every call, even if the first call that downloaded the archive already extracted it. The issue is that for some archives and on some systems, this extraction takes time (more than 10s).
Choices:
- Keep the extraction even if the archive was not re-downloaded:
- Takes time at the start of every run.
- Ensures the extracted files are correct and were not modified.
- Only extract the archive just after downloading it:
- If the extracted files are modified between runs, the next run will use those.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.measure/-/issues/70Output of metrics is confusing when not using an eval set2023-10-17T10:34:57ZYannick DAYEROutput of metrics is confusing when not using an eval setWhen evaluating one file (or multiple files without the `-e`/`--eval` option), the `metrics` option assumes this is a development set (it prints `[Min. criterion: EER ] Threshold on Development set <...>`).
This can be confusing if we w...When evaluating one file (or multiple files without the `-e`/`--eval` option), the `metrics` option assumes this is a development set (it prints `[Min. criterion: EER ] Threshold on Development set <...>`).
This can be confusing if we want to score one file on its own which could not be a development set file (e.g. to get the EER metrics of only an eval set).
suggestions:
- Change the line [script/figure.py:330](https://gitlab.idiap.ch/bob/bob.measure/-/blob/v6.1.0/src/bob/measure/script/figure.py?ref_type=tags#L330) and [342](https://gitlab.idiap.ch/bob/bob.measure/-/blob/v6.1.0/src/bob/measure/script/figure.py?ref_type=tags#L342) to something like `Threshold on file '%s': %s`, and keep the notion of `Development set` only when `self._eval == True`.
- Allow changing the set name with a new option.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/194Loading scores for the measure commands fails on non-float scores2023-08-22T08:07:17ZYannick DAYERLoading scores for the measure commands fails on non-float scores[src/bob/bio/base/score/load.py#L663](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L663) and [src/bob/bio/base/score/load.py#L650](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bi...[src/bob/bio/base/score/load.py#L663](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L663) and [src/bob/bio/base/score/load.py#L650](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L650) cast a CSV score column to float, but it could contain other things (nothing or `None`) that cannot be converted to float. This needs to be handled correctly.
Possible solution:
- Convert all non-float values to `float("nan")`, this should be handled correctly by the measure commands.https://gitlab.idiap.ch/bob/bob.pad.base/-/issues/45train set is mandatory2023-06-27T17:42:16ZYannick DAYERtrain set is mandatoryThe training is always applied in `run-pipeline`, even on non-trainable pipelines. This is fine until you consider datasets without train sets. Trying to load such a dataset results in an error.
Possible solutions:
- Detect if the data...The training is always applied in `run-pipeline`, even on non-trainable pipelines. This is fine until you consider datasets without train sets. Trying to load such a dataset results in an error.
Possible solutions:
- Detect if the database has no train set, and warn the user that no fit will be done on the pipeline;
- Add an option to skip the training (and the train set loading attempt).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/193Add a `--score-column` option in metrics and figures2023-08-22T07:59:18ZYannick DAYERAdd a `--score-column` option in metrics and figuresIn order to process score files with multiple score columns, we should have a way to select a specific column that contains the scores.
``` sh
bob bio metrics /path/to/the/file --score-column "fusion_score"
```
(Until now, we needed to...In order to process score files with multiple score columns, we should have a way to select a specific column that contains the scores.
``` sh
bob bio metrics /path/to/the/file --score-column "fusion_score"
```
(Until now, we needed to change both columns' headers to do that).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/nightlies/-/issues/64Pipeline passes even when some sub-pipeline fails2023-05-11T09:47:04ZYannick DAYERPipeline passes even when some sub-pipeline failsWhen the bob.bio.face pipeline fails (e.g. [on May the 4th](https://gitlab.idiap.ch/bob/bob.bio.face/-/pipelines/73071)), the [nightlies pipeline](https://gitlab.idiap.ch/bob/nightlies/-/pipelines/73059) does not care that it failed (it ...When the bob.bio.face pipeline fails (e.g. [on May the 4th](https://gitlab.idiap.ch/bob/bob.bio.face/-/pipelines/73071)), the [nightlies pipeline](https://gitlab.idiap.ch/bob/nightlies/-/pipelines/73059) does not care that it failed (it even marks it as succeeded), and continues executing.
I'm expecting the nightlies' pipeline to stop (or at least show that one job failed).
This is not critical as we receive a notification anyway, at the level of the package (here bob.bio.face).https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/192vuln hist command missing far_value option2023-05-01T15:11:06ZAlain KOMATYvuln hist command missing far_value optionExecuting the command:
```
bob vuln hist -c far --far-value 0.001 scores-dev.csv --output 11_hist.pdf --figsize "6,5"
```
Outputs the following error:
```
Error: No such option: --far-value Did you mean --iapmr-line?
```Executing the command:
```
bob vuln hist -c far --far-value 0.001 scores-dev.csv --output 11_hist.pdf --figsize "6,5"
```
Outputs the following error:
```
Error: No such option: --far-value Did you mean --iapmr-line?
```https://gitlab.idiap.ch/bob/docs/-/issues/13Bob citation2023-04-13T06:59:19ZHatef OTROSHIBob citationIn [Bob's webpage](https://www.idiap.ch/software/bob/), the current BibTeX is referring to the proceedings of ICML 2017, while the paper has not appeared in the proceedings of ICML 2017. Indeed, the paper is accepted in [ICML 2017 RML W...In [Bob's webpage](https://www.idiap.ch/software/bob/), the current BibTeX is referring to the proceedings of ICML 2017, while the paper has not appeared in the proceedings of ICML 2017. Indeed, the paper is accepted in [ICML 2017 RML Workshop, Reproducibility in Machine Learning](https://openreview.net/group?id=ICML.cc/2017/RML) and is available on the [Openreview platform](https://openreview.net/forum?id=BJDDItGX-¬eId=BJDDItGX-) only. Therefore, its citation needs to be fixed as follows:
```BibTeX
@inproceedings{bob2017,
author = {A. Anjos AND M. G\"unther AND T. de Freitas Pereira AND
P. Korshunov AND A. Mohammadi AND S. Marcel},
title = {Continuously Reproducing Toolchains in Pattern Recognition and
Machine Learning Experiments},
year = {2017},
month = aug,
booktitle = {ICML 2017 Reproducibility in Machine Learning Workshop},
pages={1-8},
url={https://openreview.net/forum?id=BJDDItGX-}
}
```
ping @pkorshunov @ydayerhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/98Entry-points vgg2-*-with-eval not listed in bob.bio.database group2023-03-31T14:01:51ZYannick DAYEREntry-points vgg2-*-with-eval not listed in bob.bio.database groupSome entry-points in `pyproject.toml` (notably `vgg2-short-with-eval` and `vgg2-full-with-eval`) are listed in the entry-points group `bob.bio.config` but not in `bob.bio.database`.
This leads to issues and confusion when passing the co...Some entry-points in `pyproject.toml` (notably `vgg2-short-with-eval` and `vgg2-full-with-eval`) are listed in the entry-points group `bob.bio.config` but not in `bob.bio.database`.
This leads to issues and confusion when passing the config to the `--database` option of `bob bio pipeline simple` and listing with `resources.py`.
We should (if it was not omitted for a reason) also add those config entry-points to the `bob.bio.database` entry-point group.https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/97Missing mxnet as dependency2023-03-29T16:59:00ZYannick DAYERMissing mxnet as dependencyWhen running the `arcface-insightface` baseline, an error complains that `mxnet` can not be imported.
After installing manually with `pip install mxnet`, everything works (conda did not manage to install it, though).
`mxnet` is missing...When running the `arcface-insightface` baseline, an error complains that `mxnet` can not be imported.
After installing manually with `pip install mxnet`, everything works (conda did not manage to install it, though).
`mxnet` is missing from the dependencies and dev-profile.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/191Samples lose their metadata when going through PipelineSimple2023-04-04T17:16:21ZYannick DAYERSamples lose their metadata when going through PipelineSimpleSomewhere in the pipeline, the metadata of the references and probes (age, gender, ...) are lost. They are no longer available in the score files.
Issue encountered with `bob bio pipeline simple mobio-all arcface-insightface`.
After a ...Somewhere in the pipeline, the metadata of the references and probes (age, gender, ...) are lost. They are no longer available in the score files.
Issue encountered with `bob bio pipeline simple mobio-all arcface-insightface`.
After a first investigation:
- The metadata attributes are present when creating the `Sample` objects with the `Database`.
- The `ScoreWriter` receives `Sample` objects that are already missing those attributes.
- The `Distance` BioAlgorithm seems to transform samples into bags. Maybe more investigation needed here.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.pipelines/-/issues/46Dask Client configuration not available in installed package2023-03-28T12:52:29ZYannick DAYERDask Client configuration not available in installed packageWhen using the `bob bio simple` commands, the `dask.client` entry-points are not available.
Doing `bob bio pipeline simple -H conf.py` outputs in `conf.py`:
``` python
# ----------8<----------
# dask_client = single-threaded
"""Option...When using the `bob bio simple` commands, the `dask.client` entry-points are not available.
Doing `bob bio pipeline simple -H conf.py` outputs in `conf.py`:
``` python
# ----------8<----------
# dask_client = single-threaded
"""Optional parameter: dask_client (--dask-client, -l) [default: single-threaded]
Dask client for the execution of the pipeline. Can be a `dask.client' entry point, a module name, or a path to a Python file which contains a variable named `dask_client'.
Registered entries are: []"""
# ----------8<----------
```
Tried with the package installed from conda beta; also tried with `pip install -e`.
Entry points in bob.bio.base and bob.bio.face are working. So I presume it's an issue with how we do it in this package (maybe a wrong name for the entry-point group?).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/docs/-/issues/12Links to docs in readme.md are wrong2023-10-26T13:32:15ZYannick DAYERLinks to docs in readme.md are wrongThe links to the documentation of each package in `readme.md` are wrong, as the format changed. Now the links need to point to a `sphinx` sub-directory.
The `latest` links are wrong now, but the `stable` links will become erroneous late...The links to the documentation of each package in `readme.md` are wrong, as the format changed. Now the links need to point to a `sphinx` sub-directory.
The `latest` links are wrong now, but the `stable` links will become erroneous later on (after packages are released).https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/96Annotation type XYZ is not supported.2023-03-08T10:00:15ZChristophe ECABERTAnnotation type XYZ is not supported.With recent changes to `CSVDataset`, running any baseline against `multipie` database is silently failing with the following message:
```bash
bob.bio.face.utils@2023-02-23 13:54:36,596 -- WARNING: Annotation type ('eyes-center', 'left-p...With recent changes to `CSVDataset`, running any baseline against `multipie` database is silently failing with the following message:
```bash
bob.bio.face.utils@2023-02-23 13:54:36,596 -- WARNING: Annotation type ('eyes-center', 'left-profile', 'right-profile') is not supported. Input images will be fully scaled.
```
There are two reasons for this behaviour:
- The change from `list` to `tuple` for the `MultipieDatabase.annotation_type` variable ([now](https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/master/src/bob/bio/face/database/multipie.py#L114), [then](https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/c8a6495ad85105661efa915e86a7eac7b1c2b3f6/bob/bio/face/database/multipie.py#L127))
- The oversimplified checks in [bob.bio.face.utils.py](https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/master/src/bob/bio/face/utils.py), checking only for list whereas it could potentially be any `Iterable` and not only `list`Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/95Set pytorch to run on single thread only on docker jobs2023-02-20T14:49:51ZYannick DAYERSet pytorch to run on single thread only on docker jobsTo fix the jobs on the docker runners for the CI, I added [this line](https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/d6d8e20bb73cfe4b099fedee603fad6498203d7f/src/bob/bio/face/embeddings/pytorch.py#L27) a while back.
We should:
- [x] r...To fix the jobs on the docker runners for the CI, I added [this line](https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/d6d8e20bb73cfe4b099fedee603fad6498203d7f/src/bob/bio/face/embeddings/pytorch.py#L27) a while back.
We should:
- [x] remove this line as this limits *any* work to a single thread and is not really wanted for performance reasons.
- [x] add `OMP_NUM_THREADS=1` as a variable in the CI config, either on the `.gitlab-ci.yaml` (in `bob/dev-profile`) (if possible, only for the jobs running on docker) or in the runner configuration (if possible?).André MAYORAZAndré MAYORAZhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/190SampleSet missing attributes 'key'2023-04-04T17:55:49ZChristophe ECABERTSampleSet missing attributes 'key'With the updated version of the `CSVDatabase` the `references` and `probes` are now `SampleSet` objects. Special care is taken to make sure the required attributes are defined for samples within them.
However there is no check for `Sam...With the updated version of the `CSVDatabase` the `references` and `probes` are now `SampleSet` objects. Special care is taken to make sure the required attributes are defined for samples within them.
However there is no check for `SampleSet` attributes, which is not directly an issue. But later down the pipeline, when using checkpointed experiment, the `BioAlgCheckpointWrapper` rely on the `key` attribute to run (i.e. `_enroll_sample_set(...)`).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/94Face cropping based on bounding box still requires facial landmarks / an anno...2023-02-15T15:42:45ZManuel Günthersiebenkopf@googlemail.comFace cropping based on bounding box still requires facial landmarks / an annotatorRelated to #91.
Currently, there is no easy way of cropping the face purely based on bounding boxes, i.e., without alignment based on some facial landmarks. While we have an implementation for this case in `FaceCropBoundingBox`, but thi...Related to #91.
Currently, there is no easy way of cropping the face purely based on bounding boxes, i.e., without alignment based on some facial landmarks. While we have an implementation for this case in `FaceCropBoundingBox`, but this is buggy, see #91: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/d6d8e20bb73cfe4b099fedee603fad6498203d7f/src/bob/bio/face/preprocessor/croppers.py#L312
this is not directly called from within out `face_crop_sover`: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/d6d8e20bb73cfe4b099fedee603fad6498203d7f/src/bob/bio/face/utils.py#L377
but it is only indirectly included in the `BoundingBoxAnnotatorCrop`, which uses this only for cutting out the face, and detect landmarks in the crop: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/d6d8e20bb73cfe4b099fedee603fad6498203d7f/src/bob/bio/face/preprocessor/FaceCrop.py#L305
While this is a useful use-case, another use-case would be to only extract the face based on the bounding box, without further landmark localization and alignment.
Actually, in the previous version of Bob, this was possible through (ab-)using the `FaceEyesNorm` class by providing `topleft` and `bottomright` coordinates instead.
In the current version, this is no longer possible.
I will add back an option for this.Manuel Günthersiebenkopf@googlemail.comManuel Günthersiebenkopf@googlemail.comhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/189Vulnerability analysis compares `template_id` instead of `subject_id`2023-02-13T16:35:57ZYannick DAYERVulnerability analysis compares `template_id` instead of `subject_id`To know if a score is genuine or an impostor, the fields to compare are `probe_subject_id` and `bio_ref_subject_id`.
The `vuln` commands currently use `probe_template_id` and `bio_ref_template_id`.To know if a score is genuine or an impostor, the fields to compare are `probe_subject_id` and `bio_ref_subject_id`.
The `vuln` commands currently use `probe_template_id` and `bio_ref_template_id`.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/93MTCNN comes without Non-Maximum-Suppression (NMS)2023-01-30T16:28:42ZManuel Günthersiebenkopf@googlemail.comMTCNN comes without Non-Maximum-Suppression (NMS)When running our MTCNN face detector, we get a lot of overlapping detections. Typically, these are removed with a non-maximum-suppression algorithm, see for example here: https://github.com/TropComplique/mtcnn-pytorch/blob/45b34462fc995e...When running our MTCNN face detector, we get a lot of overlapping detections. Typically, these are removed with a non-maximum-suppression algorithm, see for example here: https://github.com/TropComplique/mtcnn-pytorch/blob/45b34462fc995e6b8dbd17545b799e8c8a30026b/src/detector.py#L120 or in our TinyFaces implementation: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/de683894f9f14876293ad56390f4c34e7dd83234/src/bob/bio/face/annotator/tinyface.py#L229
However, our MTCNN implementation returns the outputs of the network unfiltered, leading to many overlapping detections: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/de683894f9f14876293ad56390f4c34e7dd83234/src/bob/bio/face/annotator/mtcnn.py#L113
When using only the first annotation as often done in our pipelines, this is not a big issue since NMS would just remove the overlapping boxes. When we need to detect more than one face in an image, on the other hand, we get a lot of repeated detections.
I would recommend to make the NMS function from TinyFaces accessible for other functions, and make use of it in MTCNN as well to filter out overlapping faces.