bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2024-03-21T10:21:20Zhttps://gitlab.idiap.ch/bob/bob.pipelines/-/issues/29jman (gridtk) like interface for submitting dask jobs2024-03-21T10:21:20ZAmir MOHAMMADIjman (gridtk) like interface for submitting dask jobsWe need:
1. A command that automatically creates a dask client for us to be used for SGE submission.
2. A history of the commands that were executed.
3. An automatic tracking of dask logs.We need:
1. A command that automatically creates a dask client for us to be used for SGE submission.
2. A history of the commands that were executed.
3. An automatic tracking of dask logs.Bob 9.0.0https://gitlab.idiap.ch/bob/bob.measure/-/issues/71No API / function to plot a histogram with scores in memory2024-02-15T13:53:03ZYannick DAYERNo API / function to plot a histogram with scores in memoryThere is no way of creating a score histogram (like `bob bio hist` from bob.bio.base) from code without saving the scores to a file.
`det`, `roc` and other plots commands have a function that allows feeding negative and positive scores ...There is no way of creating a score histogram (like `bob bio hist` from bob.bio.base) from code without saving the scores to a file.
`det`, `roc` and other plots commands have a function that allows feeding negative and positive scores as lists, but `hist` does all the plotting in the Figure class.
We should consider moving the plotting part of the `bob.measure.script.figure.Hist` class in an `hist` function in the `bob.measure.plot` module.https://gitlab.idiap.ch/bob/bob.ip.gabor/-/issues/6bob.ip.gabor in pure python2024-01-15T14:27:24ZTiago de Freitas Pereirabob.ip.gabor in pure pythonHi,
To make it easier the maintenance in the long run of Bob, we would like to get rid of all C++ packages.
By getting rid, we meant by either porting it to pure python (https://gitlab.idiap.ch/bob/bob.measure/-/merge_requests/100 , htt...Hi,
To make it easier the maintenance in the long run of Bob, we would like to get rid of all C++ packages.
By getting rid, we meant by either porting it to pure python (https://gitlab.idiap.ch/bob/bob.measure/-/merge_requests/100 , https://gitlab.idiap.ch/bob/bob.learn.linear/-/merge_requests/15 , https://gitlab.idiap.ch/bob/bob.learn.em/-/merge_requests/40) or archiving it (https://gitlab.idiap.ch/bob/bob.learn.mlp/-/issues/11 , https://gitlab.idiap.ch/bob/bob.learn.libsvm/-/issues/12 )
@mguenther, since you are the father of this package; worth the effort to port the gabor framework to pure python (for the sake of keeping our history)? Do you know any other library that does the job that `bob.ip.gabor` does?
Thankshttps://gitlab.idiap.ch/bob/bob.pipelines/-/issues/21Who is Mario?2024-01-08T13:58:44ZTiago de Freitas PereiraWho is Mario?There are several places in the code where we alias `bob.pipelines` as Mario.
Well, it was a good joke at the beginning but now we need to think if we want to keep this.There are several places in the code where we alias `bob.pipelines` as Mario.
Well, it was a good joke at the beginning but now we need to think if we want to keep this.Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.pipelines/-/issues/43Remove Bob Extension as dependency2024-01-08T13:57:32ZAndré MAYORAZRemove Bob Extension as dependencyBob extension has to be removed. It is used at three places in this package :
- [x] bob.pipelines/doc/conf.py
- To load the association list for the packages for intersphinx. We have to see if it is better to do the association between ...Bob extension has to be removed. It is used at three places in this package :
- [x] bob.pipelines/doc/conf.py
- To load the association list for the packages for intersphinx. We have to see if it is better to do the association between the package and their URL directly or any other solution.
- [x] bob.pipelines/src/bob/pipelines/distributed/sge.py
- To load the rc configuration. This can be replaced by the [exposed](https://gitlab.idiap.ch/bob/exposed) package
- [x] bob.pipelines/src/bob/pipelines/datasets.py
- To list the files and folders inside a folder or a tarball and search for files either in a file structure or in a tarball.Roadmap to the major version of Bob 12André MAYORAZAndré MAYORAZhttps://gitlab.idiap.ch/bob/bob.paper.makeup_aim/-/issues/1Please don't encourage bad practices of Bob's global configuration system.2023-12-07T08:03:25ZAmir MOHAMMADIPlease don't encourage bad practices of Bob's global configuration system.Please see:
https://www.idiap.ch/software/bob/docs/bob/docs/stable/bob/bob.extension/doc/rc.html
and https://groups.google.com/forum/#!topic/bob-devel/hA1sx5iAj9UPlease see:
https://www.idiap.ch/software/bob/docs/bob/docs/stable/bob/bob.extension/doc/rc.html
and https://groups.google.com/forum/#!topic/bob-devel/hA1sx5iAj9UKetan KotwalKetan Kotwalhttps://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/29CI: Database interfaces return empty sample list2023-11-03T09:59:05ZYannick DAYERCI: Database interfaces return empty sample list[Sometimes](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/364632) one of the CI jobs fails because a database returned an empty list when retrieving samples.
Has Mainly been seen on the `macos-arm` jobs, but always on one of both jobs...[Sometimes](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/364632) one of the CI jobs fails because a database returned an empty list when retrieving samples.
Has Mainly been seen on the `macos-arm` jobs, but always on one of both jobs (`3.9,macos-arm` or `3.10,macos-arm`).
Re-running the failed job fixes the issue.
It could be an issue with having two jobs running on the same machine. If both share the same `data` directory: if the first job finishes and cleans the database definition file, the second one has no file to read and returns an empty sample list.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/195download_file extracts the archive file every time it is called.2023-10-31T13:42:42ZYannick DAYERdownload_file extracts the archive file every time it is called.When the `extract` flag is set in [`download_file`](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/v8.0.0/src/bob/bio/base/database/utils.py?ref_type=tags#L474), the archive file is extracted at every call, even if the first call that d...When the `extract` flag is set in [`download_file`](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/v8.0.0/src/bob/bio/base/database/utils.py?ref_type=tags#L474), the archive file is extracted at every call, even if the first call that downloaded the archive already extracted it. The issue is that for some archives and on some systems, this extraction takes time (more than 10s).
Choices:
- Keep the extraction even if the archive was not re-downloaded:
- Takes time at the start of every run.
- Ensures the extracted files are correct and were not modified.
- Only extract the archive just after downloading it:
- If the extracted files are modified between runs, the next run will use those.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/docs/-/issues/12Links to docs in readme.md are wrong2023-10-26T13:32:15ZYannick DAYERLinks to docs in readme.md are wrongThe links to the documentation of each package in `readme.md` are wrong, as the format changed. Now the links need to point to a `sphinx` sub-directory.
The `latest` links are wrong now, but the `stable` links will become erroneous late...The links to the documentation of each package in `readme.md` are wrong, as the format changed. Now the links need to point to a `sphinx` sub-directory.
The `latest` links are wrong now, but the `stable` links will become erroneous later on (after packages are released).https://gitlab.idiap.ch/bob/bob.measure/-/issues/70Output of metrics is confusing when not using an eval set2023-10-17T10:34:57ZYannick DAYEROutput of metrics is confusing when not using an eval setWhen evaluating one file (or multiple files without the `-e`/`--eval` option), the `metrics` option assumes this is a development set (it prints `[Min. criterion: EER ] Threshold on Development set <...>`).
This can be confusing if we w...When evaluating one file (or multiple files without the `-e`/`--eval` option), the `metrics` option assumes this is a development set (it prints `[Min. criterion: EER ] Threshold on Development set <...>`).
This can be confusing if we want to score one file on its own which could not be a development set file (e.g. to get the EER metrics of only an eval set).
suggestions:
- Change the line [script/figure.py:330](https://gitlab.idiap.ch/bob/bob.measure/-/blob/v6.1.0/src/bob/measure/script/figure.py?ref_type=tags#L330) and [342](https://gitlab.idiap.ch/bob/bob.measure/-/blob/v6.1.0/src/bob/measure/script/figure.py?ref_type=tags#L342) to something like `Threshold on file '%s': %s`, and keep the notion of `Development set` only when `self._eval == True`.
- Allow changing the set name with a new option.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/194Loading scores for the measure commands fails on non-float scores2023-08-22T08:07:17ZYannick DAYERLoading scores for the measure commands fails on non-float scores[src/bob/bio/base/score/load.py#L663](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L663) and [src/bob/bio/base/score/load.py#L650](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bi...[src/bob/bio/base/score/load.py#L663](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L663) and [src/bob/bio/base/score/load.py#L650](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/src/bob/bio/base/score/load.py#L650) cast a CSV score column to float, but it could contain other things (nothing or `None`) that cannot be converted to float. This needs to be handled correctly.
Possible solution:
- Convert all non-float values to `float("nan")`, this should be handled correctly by the measure commands.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/193Add a `--score-column` option in metrics and figures2023-08-22T07:59:18ZYannick DAYERAdd a `--score-column` option in metrics and figuresIn order to process score files with multiple score columns, we should have a way to select a specific column that contains the scores.
``` sh
bob bio metrics /path/to/the/file --score-column "fusion_score"
```
(Until now, we needed to...In order to process score files with multiple score columns, we should have a way to select a specific column that contains the scores.
``` sh
bob bio metrics /path/to/the/file --score-column "fusion_score"
```
(Until now, we needed to change both columns' headers to do that).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.pad.base/-/issues/45train set is mandatory2023-06-27T17:42:16ZYannick DAYERtrain set is mandatoryThe training is always applied in `run-pipeline`, even on non-trainable pipelines. This is fine until you consider datasets without train sets. Trying to load such a dataset results in an error.
Possible solutions:
- Detect if the data...The training is always applied in `run-pipeline`, even on non-trainable pipelines. This is fine until you consider datasets without train sets. Trying to load such a dataset results in an error.
Possible solutions:
- Detect if the database has no train set, and warn the user that no fit will be done on the pipeline;
- Add an option to skip the training (and the train set loading attempt).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob/-/issues/274Switch to new CI/CD configuration2023-06-13T14:59:52ZYannick DAYERSwitch to new CI/CD configurationWe need to adapt this package to the new CI/CD and package format using citools:
- [ ] Modify `pyproject.toml`,
- Add information from `setup.py`,
- [ ] Empty `setup.py`,
- Leave the call to `setup()` for compatibility,
- [ ] Mo...We need to adapt this package to the new CI/CD and package format using citools:
- [ ] Modify `pyproject.toml`,
- Add information from `setup.py`,
- [ ] Empty `setup.py`,
- Leave the call to `setup()` for compatibility,
- [ ] Modify `conda/meta.yaml`,
- [ ] Remove test and documentation commands,
- [ ] Import names from `pyproject.toml`,
- [ ] Modify `.gitlab-ci.yml`,
- Use the nicer format instead of the URL,
- [ ] Add the project path to PATH in `doc/conf.py`,
- [ ] Remove `version.txt` and put the version in `pyproject.toml`.
You can look at bob.learn.em!68 for an example of a ported package.
*Note: This one might require special care compared to other ported packages as it is not a simple package*Roadmap to the major version of Bob 12https://gitlab.idiap.ch/bob/nightlies/-/issues/64Pipeline passes even when some sub-pipeline fails2023-05-11T09:47:04ZYannick DAYERPipeline passes even when some sub-pipeline failsWhen the bob.bio.face pipeline fails (e.g. [on May the 4th](https://gitlab.idiap.ch/bob/bob.bio.face/-/pipelines/73071)), the [nightlies pipeline](https://gitlab.idiap.ch/bob/nightlies/-/pipelines/73059) does not care that it failed (it ...When the bob.bio.face pipeline fails (e.g. [on May the 4th](https://gitlab.idiap.ch/bob/bob.bio.face/-/pipelines/73071)), the [nightlies pipeline](https://gitlab.idiap.ch/bob/nightlies/-/pipelines/73059) does not care that it failed (it even marks it as succeeded), and continues executing.
I'm expecting the nightlies' pipeline to stop (or at least show that one job failed).
This is not critical as we receive a notification anyway, at the level of the package (here bob.bio.face).https://gitlab.idiap.ch/bob/nightlies/-/issues/62Nightlies on stages2023-05-11T09:17:24ZTiago de Freitas PereiraNightlies on stagesHi,
I think would be better to split this nightly builds in stages where in each stage we have a set of key packages.
For instance, in the first stage we could have the core packages.
- bob/bob.extension
- bob/bob.io.base
- bob/bob....Hi,
I think would be better to split this nightly builds in stages where in each stage we have a set of key packages.
For instance, in the first stage we could have the core packages.
- bob/bob.extension
- bob/bob.io.base
- bob/bob.measure
- bob/bob.learn.em
- bob/bob.pipelines
So, if one of these five packages (only five :-) ) fails we need to act immediately.
In the second, third or N-th stage we could pack a set of packages to be nightly executed. For instance, we could have a nightly for biometrics with the packages:
- bob/bob.bio.base
- bob/bob.bio.face
- bob/bob.bio.video
- bob/bob.bio.vein
- bob/bob.bio.spear
- bob/bob.pad.base
- bob/bob.pad.face
If one of these packages fails, we don't need to act urgently.
what do you think?
PS: this was @andre.anjos ideahttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/192vuln hist command missing far_value option2023-05-01T15:11:06ZAlain KOMATYvuln hist command missing far_value optionExecuting the command:
```
bob vuln hist -c far --far-value 0.001 scores-dev.csv --output 11_hist.pdf --figsize "6,5"
```
Outputs the following error:
```
Error: No such option: --far-value Did you mean --iapmr-line?
```Executing the command:
```
bob vuln hist -c far --far-value 0.001 scores-dev.csv --output 11_hist.pdf --figsize "6,5"
```
Outputs the following error:
```
Error: No such option: --far-value Did you mean --iapmr-line?
```https://gitlab.idiap.ch/bob/docs/-/issues/13Bob citation2023-04-13T06:59:19ZHatef OTROSHIBob citationIn [Bob's webpage](https://www.idiap.ch/software/bob/), the current BibTeX is referring to the proceedings of ICML 2017, while the paper has not appeared in the proceedings of ICML 2017. Indeed, the paper is accepted in [ICML 2017 RML W...In [Bob's webpage](https://www.idiap.ch/software/bob/), the current BibTeX is referring to the proceedings of ICML 2017, while the paper has not appeared in the proceedings of ICML 2017. Indeed, the paper is accepted in [ICML 2017 RML Workshop, Reproducibility in Machine Learning](https://openreview.net/group?id=ICML.cc/2017/RML) and is available on the [Openreview platform](https://openreview.net/forum?id=BJDDItGX-¬eId=BJDDItGX-) only. Therefore, its citation needs to be fixed as follows:
```BibTeX
@inproceedings{bob2017,
author = {A. Anjos AND M. G\"unther AND T. de Freitas Pereira AND
P. Korshunov AND A. Mohammadi AND S. Marcel},
title = {Continuously Reproducing Toolchains in Pattern Recognition and
Machine Learning Experiments},
year = {2017},
month = aug,
booktitle = {ICML 2017 Reproducibility in Machine Learning Workshop},
pages={1-8},
url={https://openreview.net/forum?id=BJDDItGX-}
}
```
ping @pkorshunov @ydayerhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/190SampleSet missing attributes 'key'2023-04-04T17:55:49ZChristophe ECABERTSampleSet missing attributes 'key'With the updated version of the `CSVDatabase` the `references` and `probes` are now `SampleSet` objects. Special care is taken to make sure the required attributes are defined for samples within them.
However there is no check for `Sam...With the updated version of the `CSVDatabase` the `references` and `probes` are now `SampleSet` objects. Special care is taken to make sure the required attributes are defined for samples within them.
However there is no check for `SampleSet` attributes, which is not directly an issue. But later down the pipeline, when using checkpointed experiment, the `BioAlgCheckpointWrapper` rely on the `key` attribute to run (i.e. `_enroll_sample_set(...)`).Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/191Samples lose their metadata when going through PipelineSimple2023-04-04T17:16:21ZYannick DAYERSamples lose their metadata when going through PipelineSimpleSomewhere in the pipeline, the metadata of the references and probes (age, gender, ...) are lost. They are no longer available in the score files.
Issue encountered with `bob bio pipeline simple mobio-all arcface-insightface`.
After a ...Somewhere in the pipeline, the metadata of the references and probes (age, gender, ...) are lost. They are no longer available in the score files.
Issue encountered with `bob bio pipeline simple mobio-all arcface-insightface`.
After a first investigation:
- The metadata attributes are present when creating the `Sample` objects with the `Database`.
- The `ScoreWriter` receives `Sample` objects that are already missing those attributes.
- The `Distance` BioAlgorithm seems to transform samples into bags. Maybe more investigation needed here.Yannick DAYERYannick DAYER