bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2021-12-13T08:13:16Zhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/167Algorithms with training that requires split by class don't seem to work2021-12-13T08:13:16ZManuel Günthersiebenkopf@googlemail.comAlgorithms with training that requires split by class don't seem to workWhen running a small baseline algorithm, such as `lda`, it seems that the required classes for the training samples is not forwarded to the training algorithm:
```
$ bob bio pipelines vanilla-biometrics -vv atnt lda
...
File ".../bob...When running a small baseline algorithm, such as `lda`, it seems that the required classes for the training samples is not forwarded to the training algorithm:
```
$ bob bio pipelines vanilla-biometrics -vv atnt lda
...
File ".../bob.bio.base/bob/bio/base/transformers/algorithm.py", line 62, in fit
training_data = split_X_by_y(X, y)
File ".../bob.bio.base/bob/bio/base/transformers/__init__.py", line 6, in split_X_by_y
for x1, y1 in zip(X, y):
TypeError: 'NoneType' object is not iterable
```
I have checked what is going on, and it seems that `y=None` in: https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/1c3f542ee4d77592146ddc54aa8a51194a853745/bob/bio/base/transformers/__init__.py#L4
called by:
https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/1c3f542ee4d77592146ddc54aa8a51194a853745/bob/bio/base/transformers/algorithm.py#L61
Unfortunately, I cannot trace the issue back further since my experience in debugging `dask` is very limited.
Maybe we should allow to run the pipeline without `dask` -- as far as I understood, the dask-pipeline is only a wrapper around the whole pipeline. Is it possible to skip using the `dask` wrapper and run everything local in a single thread? This would make debugging much easier.
Actually, I wanted to try out the above pipeline to debug my `dask` setup, which does not work.Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.demographics/-/issues/2A lot of work to be done in this package2021-11-26T20:18:30ZTiago de Freitas PereiraA lot of work to be done in this packageThis package has the purpose to be the main back for fairness in biometrics.
However, several bits are missing here.
Follow below a TODO list.
- [ ] Proper user guide and documentation
- [ ] Implementation of a pipeline that does PlatSc...This package has the purpose to be the main back for fairness in biometrics.
However, several bits are missing here.
Follow below a TODO list.
- [ ] Proper user guide and documentation
- [ ] Implementation of a pipeline that does PlatScalling and calibration by group
- [ ] Port the regularization strategies that it's implemented on TF to PyTorch.Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/166Score normalization pipeline needs some redesign2022-01-13T10:03:34ZTiago de Freitas PereiraScore normalization pipeline needs some redesign`bob.bio.base` implements a pipeline that does several types of score normalization in one shot:
- Z-Norm
- T-Norm
- S-Norm
- ZT-Norm
- Some variations of the adaptative norm.
Although logic (they are all variations of the same thi...`bob.bio.base` implements a pipeline that does several types of score normalization in one shot:
- Z-Norm
- T-Norm
- S-Norm
- ZT-Norm
- Some variations of the adaptative norm.
Although logic (they are all variations of the same thing), this structure doesn't seem to scale to datasets where the number of comparisons explodes to millions of comparisons.
I often face `MemoryError` issues that are super tough to track down (dask memory error).
Furthermore, the code is a bit convoluted. I think we need to break this down into small pieces.Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/165Move some features from bob.measure to bob.bio.base2021-11-26T17:45:12ZTiago de Freitas PereiraMove some features from bob.measure to bob.bio.baseAs we've discussed in the last bob meeting we decided to move some biometric specific features from `bob.measure` to `bob.bio.base`
The candidate features are the following (including the plot extension of it):
- bob.measure.cmc
- bo...As we've discussed in the last bob meeting we decided to move some biometric specific features from `bob.measure` to `bob.bio.base`
The candidate features are the following (including the plot extension of it):
- bob.measure.cmc
- bob.measure.dir
- bob.measure.epc
- ANYTHING ELSE?
Depends on: https://gitlab.idiap.ch/bob/bob.measure/-/merge_requests/103
ping @amohammadi @lcolbois @ydayer
Thankshttps://gitlab.idiap.ch/bob/bob.measure/-/issues/66Issue with ROC curve2021-11-29T11:24:22ZTiago de Freitas PereiraIssue with ROC curveHi @andre.anjos,
You've mentioned in our last Bob meeting that there's an issue with the ROC plot in some special cases.
Do you have any file containing the true values and prediction scores that triggers the issue?
ThanksHi @andre.anjos,
You've mentioned in our last Bob meeting that there's an issue with the ROC plot in some special cases.
Do you have any file containing the true values and prediction scores that triggers the issue?
Thankshttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/164Temporary files and caches are written into the result directory2021-12-14T17:58:32ZManuel Günthersiebenkopf@googlemail.comTemporary files and caches are written into the result directoryIn the old version, we had two different directories to store elements: the `result_directory` and the `temp_directory`. These two directories where there for a purpose: anything inside of `temp` could be easily removed after the experim...In the old version, we had two different directories to store elements: the `result_directory` and the `temp_directory`. These two directories where there for a purpose: anything inside of `temp` could be easily removed after the experiments have finished, while important results were stored in the `results` directory. This separation also allowed to have the temporary files on a local disk -- with much faster access and without backup -- and only the result files in a directory with backups.
Unfortunately, this split has gone in the new version -- for no obvious reasons other than laziness IMHO. Is there any possibility to have some kind of mechanism to have files in `tmp` and cached files in `sampleswrapper`, `biometric_references` and `scores` to be placed in a different directory than the `--output`? Or is there any particular reason that you only want to have a single directory for all output that I am overlooking here?https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/163`annotate` does not create files for empty annotations2021-11-24T10:07:55ZYannick DAYER`annotate` does not create files for empty annotationsMissing annotations files lead to errors when trying to load them later on.
Files are not created since we use `CheckpointWrapper` to save them and it skips samples with `output_attribute == None`.
Need to change the save system of [th...Missing annotations files lead to errors when trying to load them later on.
Files are not created since we use `CheckpointWrapper` to save them and it skips samples with `output_attribute == None`.
Need to change the save system of [the `annotate` script](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/bob/bio/base/script/annotate.py#L106) to not use `CheckpointWrapper`.
Expected behavior: Failed execution of the annotator on a sample still creates a file for that sample with `"null"` or another expected value.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/92Test for yum install does not work if builder tag **contains** `docker`2021-11-30T14:56:56ZAndré AnjosTest for yum install does not work if builder tag **contains** `docker`The test verifies the following:
```python
if "docker" in os.environ.get("CI_RUNNER_TAGS", "") and os.path.exists(
yum_requirements_file
):
...
```
The CI_RUNNER_TAGS for the shell builder says `docker-build`, a...The test verifies the following:
```python
if "docker" in os.environ.get("CI_RUNNER_TAGS", "") and os.path.exists(
yum_requirements_file
):
...
```
The CI_RUNNER_TAGS for the shell builder says `docker-build`, and therefore this test will also succeed then, whereas it shouldn't.
Failing pipeline: https://gitlab.idiap.ch/beat/beat.nightlies/-/jobs/250244
The solution would be to split the tags, and compare to "exactly" `docker` and not to just contain it.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/71BobIpTinyface returns a empty list list on failure but is tested for None2021-11-22T18:34:38ZManuel Günthersiebenkopf@googlemail.comBobIpTinyface returns a empty list list on failure but is tested for NoneThere is a mismatch between how the error case of not detecting any face is raised (empty list):
https://gitlab.idiap.ch/bob/bob.ip.facedetect/-/blob/db3756548da07918e7b9b4a8c2cd3dcf7783f810/bob/ip/facedetect/tinyface.py#L228
and handled...There is a mismatch between how the error case of not detecting any face is raised (empty list):
https://gitlab.idiap.ch/bob/bob.ip.facedetect/-/blob/db3756548da07918e7b9b4a8c2cd3dcf7783f810/bob/ip/facedetect/tinyface.py#L228
and handled (tested for `None`):
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/ead8c069bafb4024dc15c5df7fdc878aec8bd5f0/bob/bio/face/annotator/bobiptinyface.py#L39Manuel Günthersiebenkopf@googlemail.comManuel Günthersiebenkopf@googlemail.comhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/70BoundingBoxAnnotatorCrop ignores the selected bounding boxes2021-12-14T14:36:36ZManuel Günthersiebenkopf@googlemail.comBoundingBoxAnnotatorCrop ignores the selected bounding boxesThe `BoundingBoxAnnotatorCrop` is nice in the sense that it can even work when no face was detected by simply using the bounding box. While the `"topleft"` and `"bottomright"` are required to be specified, they are however ignored in th...The `BoundingBoxAnnotatorCrop` is nice in the sense that it can even work when no face was detected by simply using the bounding box. While the `"topleft"` and `"bottomright"` are required to be specified, they are however ignored in the constructor:
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/ead8c069bafb4024dc15c5df7fdc878aec8bd5f0/bob/bio/face/preprocessor/FaceCrop.py#L542
Instead, the face is cropped and simply scaled to the required size:
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/ead8c069bafb4024dc15c5df7fdc878aec8bd5f0/bob/bio/face/preprocessor/FaceCrop.py#L616
Finally, the `fixed_positions` are not respected by the annotator, so it is impossible to use it with a dataset that does not provide `topleft` and `bottomright` annotations.Manuel Günthersiebenkopf@googlemail.comManuel Günthersiebenkopf@googlemail.comhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/69Some databases ignore fixed positions2021-11-22T18:28:10ZManuel Günthersiebenkopf@googlemail.comSome databases ignore fixed positionsIn some databases, the `fixed_positions` flags are ignored. While it might make sens for some of them, for others (such as LFW) it might be interesting to set some fixed positions since faces are in the center of all images.In some databases, the `fixed_positions` flags are ignored. While it might make sens for some of them, for others (such as LFW) it might be interesting to set some fixed positions since faces are in the center of all images.Manuel Günthersiebenkopf@googlemail.comManuel Günthersiebenkopf@googlemail.comhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/68Pytorch device is not followed in embeddings2021-11-22T18:07:05ZManuel Günthersiebenkopf@googlemail.comPytorch device is not followed in embeddingsWhen setting a specific device to use for feature extraction in pytorch, the features are still extracted using device "cuda" (i.e., the first GPU).When setting a specific device to use for feature extraction in pytorch, the features are still extracted using device "cuda" (i.e., the first GPU).Manuel Günthersiebenkopf@googlemail.comManuel Günthersiebenkopf@googlemail.comhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/67The LFW annotations provided on the biometric resources cannot be read by the...2021-11-22T18:41:53ZManuel Günthersiebenkopf@googlemail.comThe LFW annotations provided on the biometric resources cannot be read by the current LFW database implementationIn the current implementation of the LFW database, there are two ways on how to read the annotations:
- `funneled`: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/ead8c069bafb4024dc15c5df7fdc878aec8bd5f0/bob/bio/face/database/lfw.py#L1...In the current implementation of the LFW database, there are two ways on how to read the annotations:
- `funneled`: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/ead8c069bafb4024dc15c5df7fdc878aec8bd5f0/bob/bio/face/database/lfw.py#L151
- `idiap`: https://gitlab.idiap.ch/bob/bob.db.lfw/-/blob/5ac22c5b77aae971de6b73cbe23f26d6a5632072/bob/db/lfw/models.py#L81
However, it is not clear how we can obtain the annotations in these formats. On the other hand, the Biometric Resources page https://www.idiap.ch/webarchives/sites/www.idiap.ch/resource/biometric/ provides annotations in a format that previously has been termed `named`. Unfortunately, there is no implementation for reading this format.https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/66LFW directories are non-standard2022-02-03T12:53:53ZManuel Günthersiebenkopf@googlemail.comLFW directories are non-standardIt appears that the directory structure for the LFW dataset is set to an idiap-specific location. By default, there is no relative path like this:
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/ead8c069bafb4024dc15c5df7fdc878aec8bd5f0/b...It appears that the directory structure for the LFW dataset is set to an idiap-specific location. By default, there is no relative path like this:
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/ead8c069bafb4024dc15c5df7fdc878aec8bd5f0/bob/bio/face/database/lfw.py#L87
When downloading the images from the LFW web page, they get extracted into directories `lfw` or `lfw_funneled` (for the funneled version). In my eyes it would be more sensible to used these directories instead, i.e., use `lfw` when no annotation directory is specified, and use `lfw_funneled` when annotations are provided.
Another way would be to have two distinct entries for `lfw-view2`, such as `lfw-view2-aligned` (funneled images and annotations) and `lfw-view2-raw` (original images, use detectors).https://gitlab.idiap.ch/bob/bob.devtools/-/issues/91`bdt` assumes `mamba` to be in base environment2021-11-09T16:40:23ZManuel Günthersiebenkopf@googlemail.com`bdt` assumes `mamba` to be in base environmentI have just installed `bob.devtools` to develop a local package. To not screw up my base environment, I have installed `mamba` and `bob.devtools` in a separate conda environment:
```
conda create -n bob-devel python=3.8 mamba -c conda-f...I have just installed `bob.devtools` to develop a local package. To not screw up my base environment, I have installed `mamba` and `bob.devtools` in a separate conda environment:
```
conda create -n bob-devel python=3.8 mamba -c conda-forge
conda activate bob-devel
mamba install bob.devtools -c https://www.idiap.ch/software/bob/conda/label/beta/ -c conda-forge
```
When I now want to build `my-package`, I get the following error message:
```
(bob-devel)$ bdt create -vv my-package
...
Traceback (most recent call last):
File ".../miniconda/envs/ijbc/bin/bdt", line 11, in <module>
sys.exit(main())
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/bob/devtools/scripts/bdt.py", line 43, in _decorator
value = view_func(*args, **kwargs)
File "/.../miniconda/envs/ijbc/lib/python3.8/site-packages/bob/devtools/scripts/create.py", line 272, in create
conda_create(
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/bob/devtools/build.py", line 413, in conda_create
envdir = get_env_directory(conda, name)
File ".../miniconda/envs/ijbc/lib/python3.8/site-packages/bob/devtools/build.py", line 362, in get_env_directory
output = subprocess.check_output(cmd)
File ".../miniconda/envs/ijbc/lib/python3.8/subprocess.py", line 415, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File ".../miniconda/envs/ijbc/lib/python3.8/subprocess.py", line 493, in run
with Popen(*popenargs, **kwargs) as process:
File ".../miniconda/envs/ijbc/lib/python3.8/subprocess.py", line 858, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File ".../miniconda/envs/ijbc/lib/python3.8/subprocess.py", line 1704, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '.../miniconda/bin/mamba'
```
This seems that the script looks for `mamba` in the base environment, but actually it is only in my current environment:
```
(bob-devel)$ which mamba
.../miniconda/envs/bob-devel/bin/mamba
```
Is there a way to use `mamba` from my current environment?https://gitlab.idiap.ch/bob/bob.io.audio/-/issues/9package installed with conda is broken2021-12-13T10:52:37ZAmir MOHAMMADIpackage installed with conda is broken```
bob10/lib/python3.8/site-packages/bob/io/audio/__init__.py", line 4, in <module>
from ._library import *
ImportError: libsox.so.3: cannot open shared object file: No such file or directory
```
Also I see that pytest never ran bo...```
bob10/lib/python3.8/site-packages/bob/io/audio/__init__.py", line 4, in <module>
from ._library import *
ImportError: libsox.so.3: cannot open shared object file: No such file or directory
```
Also I see that pytest never ran bob.io.audio tests in https://gitlab.idiap.ch/bob/bob/-/jobs/249166/rawAmir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.paper.8years/-/issues/2Things to be done in this paper package2022-08-11T12:58:08ZTiago de Freitas PereiraThings to be done in this paper package- [ ] Add some decent documentation on how to bootstrap the experiments and the special plots
- [ ] Make a dashboard on with ALL our baselines in a sphinx format
- [ ] Improve the experiment trigger use use resources instead of looking ...- [ ] Add some decent documentation on how to bootstrap the experiments and the special plots
- [ ] Make a dashboard on with ALL our baselines in a sphinx format
- [ ] Improve the experiment trigger use use resources instead of looking at configuration fileshttps://gitlab.idiap.ch/bob/bob.pipelines/-/issues/37dask_jobqueue > 0.7.2 changed the API...2021-11-30T18:25:54ZTiago de Freitas Pereiradask_jobqueue > 0.7.2 changed the API..... and this is breaking `SGEIdiapJob`
`**kwargs` was removed from here
https://github.com/dask/dask-jobqueue/blob/0.7.2/dask_jobqueue/core.py#L132.. and this is breaking `SGEIdiapJob`
`**kwargs` was removed from here
https://github.com/dask/dask-jobqueue/blob/0.7.2/dask_jobqueue/core.py#L132Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/90Stable deployment failed2021-11-03T17:05:04ZAmir MOHAMMADIStable deployment failedJob [#248688](https://gitlab.idiap.ch/bob/bob.ip.base/-/jobs/248688) failed for df03050b446a54b269d95e4f9c306fda35f95375:Job [#248688](https://gitlab.idiap.ch/bob/bob.ip.base/-/jobs/248688) failed for df03050b446a54b269d95e4f9c306fda35f95375:https://gitlab.idiap.ch/bob/bob.io.stream/-/issues/5docs not formatted correctly in bob/docs2021-11-25T10:38:15ZAmir MOHAMMADIdocs not formatted correctly in bob/docsplease see the sidebar in https://www.idiap.ch/software/bob/docs/bob/docs/master/index.html
![image](/uploads/dfb0d6e77b3151f7eae1d79a3cedf25e/image.png)please see the sidebar in https://www.idiap.ch/software/bob/docs/bob/docs/master/index.html
![image](/uploads/dfb0d6e77b3151f7eae1d79a3cedf25e/image.png)Vincent POLLETVincent POLLET