bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2023-04-04T17:16:21Zhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/191Samples lose their metadata when going through PipelineSimple2023-04-04T17:16:21ZYannick DAYERSamples lose their metadata when going through PipelineSimpleSomewhere in the pipeline, the metadata of the references and probes (age, gender, ...) are lost. They are no longer available in the score files.
Issue encountered with `bob bio pipeline simple mobio-all arcface-insightface`.
After a ...Somewhere in the pipeline, the metadata of the references and probes (age, gender, ...) are lost. They are no longer available in the score files.
Issue encountered with `bob bio pipeline simple mobio-all arcface-insightface`.
After a first investigation:
- The metadata attributes are present when creating the `Sample` objects with the `Database`.
- The `ScoreWriter` receives `Sample` objects that are already missing those attributes.
- The `Distance` BioAlgorithm seems to transform samples into bags. Maybe more investigation needed here.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/97Missing mxnet as dependency2023-03-29T16:59:00ZYannick DAYERMissing mxnet as dependencyWhen running the `arcface-insightface` baseline, an error complains that `mxnet` can not be imported.
After installing manually with `pip install mxnet`, everything works (conda did not manage to install it, though).
`mxnet` is missing...When running the `arcface-insightface` baseline, an error complains that `mxnet` can not be imported.
After installing manually with `pip install mxnet`, everything works (conda did not manage to install it, though).
`mxnet` is missing from the dependencies and dev-profile.https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/98Entry-points vgg2-*-with-eval not listed in bob.bio.database group2023-03-31T14:01:51ZYannick DAYEREntry-points vgg2-*-with-eval not listed in bob.bio.database groupSome entry-points in `pyproject.toml` (notably `vgg2-short-with-eval` and `vgg2-full-with-eval`) are listed in the entry-points group `bob.bio.config` but not in `bob.bio.database`.
This leads to issues and confusion when passing the co...Some entry-points in `pyproject.toml` (notably `vgg2-short-with-eval` and `vgg2-full-with-eval`) are listed in the entry-points group `bob.bio.config` but not in `bob.bio.database`.
This leads to issues and confusion when passing the config to the `--database` option of `bob bio pipeline simple` and listing with `resources.py`.
We should (if it was not omitted for a reason) also add those config entry-points to the `bob.bio.database` entry-point group.https://gitlab.idiap.ch/bob/docs/-/issues/13Bob citation2023-04-13T06:59:19ZHatef OTROSHIBob citationIn [Bob's webpage](https://www.idiap.ch/software/bob/), the current BibTeX is referring to the proceedings of ICML 2017, while the paper has not appeared in the proceedings of ICML 2017. Indeed, the paper is accepted in [ICML 2017 RML W...In [Bob's webpage](https://www.idiap.ch/software/bob/), the current BibTeX is referring to the proceedings of ICML 2017, while the paper has not appeared in the proceedings of ICML 2017. Indeed, the paper is accepted in [ICML 2017 RML Workshop, Reproducibility in Machine Learning](https://openreview.net/group?id=ICML.cc/2017/RML) and is available on the [Openreview platform](https://openreview.net/forum?id=BJDDItGX-¬eId=BJDDItGX-) only. Therefore, its citation needs to be fixed as follows:
```BibTeX
@inproceedings{bob2017,
author = {A. Anjos AND M. G\"unther AND T. de Freitas Pereira AND
P. Korshunov AND A. Mohammadi AND S. Marcel},
title = {Continuously Reproducing Toolchains in Pattern Recognition and
Machine Learning Experiments},
year = {2017},
month = aug,
booktitle = {ICML 2017 Reproducibility in Machine Learning Workshop},
pages={1-8},
url={https://openreview.net/forum?id=BJDDItGX-}
}
```
ping @pkorshunov @ydayerhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/192vuln hist command missing far_value option2023-05-01T15:11:06ZAlain KOMATYvuln hist command missing far_value optionExecuting the command:
```
bob vuln hist -c far --far-value 0.001 scores-dev.csv --output 11_hist.pdf --figsize "6,5"
```
Outputs the following error:
```
Error: No such option: --far-value Did you mean --iapmr-line?
```Executing the command:
```
bob vuln hist -c far --far-value 0.001 scores-dev.csv --output 11_hist.pdf --figsize "6,5"
```
Outputs the following error:
```
Error: No such option: --far-value Did you mean --iapmr-line?
```https://gitlab.idiap.ch/bob/nightlies/-/issues/64Pipeline passes even when some sub-pipeline fails2023-05-11T09:47:04ZYannick DAYERPipeline passes even when some sub-pipeline failsWhen the bob.bio.face pipeline fails (e.g. [on May the 4th](https://gitlab.idiap.ch/bob/bob.bio.face/-/pipelines/73071)), the [nightlies pipeline](https://gitlab.idiap.ch/bob/nightlies/-/pipelines/73059) does not care that it failed (it ...When the bob.bio.face pipeline fails (e.g. [on May the 4th](https://gitlab.idiap.ch/bob/bob.bio.face/-/pipelines/73071)), the [nightlies pipeline](https://gitlab.idiap.ch/bob/nightlies/-/pipelines/73059) does not care that it failed (it even marks it as succeeded), and continues executing.
I'm expecting the nightlies' pipeline to stop (or at least show that one job failed).
This is not critical as we receive a notification anyway, at the level of the package (here bob.bio.face).https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/193Add a `--score-column` option in metrics and figures2023-08-22T07:59:18ZYannick DAYERAdd a `--score-column` option in metrics and figuresIn order to process score files with multiple score columns, we should have a way to select a specific column that contains the scores.
``` sh
bob bio metrics /path/to/the/file --score-column "fusion_score"
```
(Until now, we needed to...In order to process score files with multiple score columns, we should have a way to select a specific column that contains the scores.
``` sh
bob bio metrics /path/to/the/file --score-column "fusion_score"
```
(Until now, we needed to change both columns' headers to do that).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.pad.base/-/issues/45train set is mandatory2023-06-27T17:42:16ZYannick DAYERtrain set is mandatoryThe training is always applied in `run-pipeline`, even on non-trainable pipelines. This is fine until you consider datasets without train sets. Trying to load such a dataset results in an error.
Possible solutions:
- Detect if the data...The training is always applied in `run-pipeline`, even on non-trainable pipelines. This is fine until you consider datasets without train sets. Trying to load such a dataset results in an error.
Possible solutions:
- Detect if the database has no train set, and warn the user that no fit will be done on the pipeline;
- Add an option to skip the training (and the train set loading attempt).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/194Loading scores for the measure commands fails on non-float scores2023-08-22T08:07:17ZYannick DAYERLoading scores for the measure commands fails on non-float scores[src/bob/bio/base/score/load.py#L663](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L663) and [src/bob/bio/base/score/load.py#L650](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bi...[src/bob/bio/base/score/load.py#L663](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L663) and [src/bob/bio/base/score/load.py#L650](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L650) cast a CSV score column to float, but it could contain other things (nothing or `None`) that cannot be converted to float. This needs to be handled correctly.
Possible solution:
- Convert all non-float values to `float("nan")`, this should be handled correctly by the measure commands.https://gitlab.idiap.ch/bob/bob.measure/-/issues/70Output of metrics is confusing when not using an eval set2023-10-17T10:34:57ZYannick DAYEROutput of metrics is confusing when not using an eval setWhen evaluating one file (or multiple files without the `-e`/`--eval` option), the `metrics` option assumes this is a development set (it prints `[Min. criterion: EER ] Threshold on Development set <...>`).
This can be confusing if we w...When evaluating one file (or multiple files without the `-e`/`--eval` option), the `metrics` option assumes this is a development set (it prints `[Min. criterion: EER ] Threshold on Development set <...>`).
This can be confusing if we want to score one file on its own which could not be a development set file (e.g. to get the EER metrics of only an eval set).
suggestions:
- Change the line [script/figure.py:330](https://gitlab.idiap.ch/bob/bob.measure/-/blob/v6.1.0/src/bob/measure/script/figure.py?ref_type=tags#L330) and [342](https://gitlab.idiap.ch/bob/bob.measure/-/blob/v6.1.0/src/bob/measure/script/figure.py?ref_type=tags#L342) to something like `Threshold on file '%s': %s`, and keep the notion of `Development set` only when `self._eval == True`.
- Allow changing the set name with a new option.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/195download_file extracts the archive file every time it is called.2023-10-31T13:42:42ZYannick DAYERdownload_file extracts the archive file every time it is called.When the `extract` flag is set in [`download_file`](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/v8.0.0/src/bob/bio/base/database/utils.py?ref_type=tags#L474), the archive file is extracted at every call, even if the first call that d...When the `extract` flag is set in [`download_file`](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/v8.0.0/src/bob/bio/base/database/utils.py?ref_type=tags#L474), the archive file is extracted at every call, even if the first call that downloaded the archive already extracted it. The issue is that for some archives and on some systems, this extraction takes time (more than 10s).
Choices:
- Keep the extraction even if the archive was not re-downloaded:
- Takes time at the start of every run.
- Ensures the extracted files are correct and were not modified.
- Only extract the archive just after downloading it:
- If the extracted files are modified between runs, the next run will use those.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/29CI: Database interfaces return empty sample list2023-11-03T09:59:05ZYannick DAYERCI: Database interfaces return empty sample list[Sometimes](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/364632) one of the CI jobs fails because a database returned an empty list when retrieving samples.
Has Mainly been seen on the `macos-arm` jobs, but always on one of both jobs...[Sometimes](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/364632) one of the CI jobs fails because a database returned an empty list when retrieving samples.
Has Mainly been seen on the `macos-arm` jobs, but always on one of both jobs (`3.9,macos-arm` or `3.10,macos-arm`).
Re-running the failed job fixes the issue.
It could be an issue with having two jobs running on the same machine. If both share the same `data` directory: if the first job finishes and cleans the database definition file, the second one has no file to read and returns an empty sample list.https://gitlab.idiap.ch/bob/bob.measure/-/issues/71No API / function to plot a histogram with scores in memory2024-02-15T13:53:03ZYannick DAYERNo API / function to plot a histogram with scores in memoryThere is no way of creating a score histogram (like `bob bio hist` from bob.bio.base) from code without saving the scores to a file.
`det`, `roc` and other plots commands have a function that allows feeding negative and positive scores ...There is no way of creating a score histogram (like `bob bio hist` from bob.bio.base) from code without saving the scores to a file.
`det`, `roc` and other plots commands have a function that allows feeding negative and positive scores as lists, but `hist` does all the plotting in the Figure class.
We should consider moving the plotting part of the `bob.measure.script.figure.Hist` class in an `hist` function in the `bob.measure.plot` module.