bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2022-08-10T09:42:39Zhttps://gitlab.idiap.ch/bob/bob.paper.8years/-/issues/4Instructions for GBU plots are missing in the README2022-08-10T09:42:39ZManuel Günthersiebenkopf@googlemail.comInstructions for GBU plots are missing in the READMEThe README lists 6 datasets to be used, but provides commands only for 5 of them. As far as I can see, the GBU dataset is missing.The README lists 6 datasets to be used, but provides commands only for 5 of them. As far as I can see, the GBU dataset is missing.Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/27`all_samples` in `full` protocol of utfvp dataset2022-08-10T08:28:12ZHatef OTROSHI`all_samples` in `full` protocol of utfvp datasetHi,
It seems that `all_samples` in `full` protocol of the utfvp dataset has a problem and raises an issue.
Here is a sample code:
```
from bob.bio.vein.database.utfvp import UtfvpDatabase
database = UtfvpDatabase(protocol="full")
datab...Hi,
It seems that `all_samples` in `full` protocol of the utfvp dataset has a problem and raises an issue.
Here is a sample code:
```
from bob.bio.vein.database.utfvp import UtfvpDatabase
database = UtfvpDatabase(protocol="full")
database.all_samples()
```
and here is the error:
```
packages/bob/bio/base/database/csv_dataset.py", line 494, in all_samples
samples = samples + self.background_model_samples()
File "..../lib/python3.9/site-packages/bob/bio/base/database/csv_dataset.py", line 381, in background_model_samples
self.csv_to_sample_loader.transform(self.train_csv)
File "..../lib/python3.9/site-packages/sklearn/pipeline.py", line 635, in transform
Xt = transform.transform(Xt)
File "..../lib/python3.9/site-packages/bob/pipelines/sample_loaders.py", line 84, in transform
X.seek(0)
AttributeError: 'NoneType' object has no attribute 'seek'
```https://gitlab.idiap.ch/bob/bob.paper.8years/-/issues/3Move plots to bob.bio.face2022-08-08T10:59:50ZTiago de Freitas PereiraMove plots to bob.bio.faceHi @mguenther,
We've made in this package several customized plots.
Can I move those customized plots to `bob.bio.face`?
ThanksHi @mguenther,
We've made in this package several customized plots.
Can I move those customized plots to `bob.bio.face`?
Thankshttps://gitlab.idiap.ch/bob/bob.bio.spear/-/issues/43SpeechbrainEmbeddings fails when run in dask for the first time2022-07-21T09:09:34ZYannick DAYERSpeechbrainEmbeddings fails when run in dask for the first timeWhen running `speechbrain-ecapa-voxceleb` for the first time, it fails if on dask.
Multiple workers are calling load_model at the same time and as the model files do not exist (first time running the pipeline), speechbrain tries to down...When running `speechbrain-ecapa-voxceleb` for the first time, it fails if on dask.
Multiple workers are calling load_model at the same time and as the model files do not exist (first time running the pipeline), speechbrain tries to download them from huggingface. It uses a cache in `~/.cache/huggingface`. Error rise as multiple worker access simultaneousely to the same files.
Workaround: The first time you run `speechbrain-ecapa-voxceleb`, do it without the `-l ...` option, at least until the files are downloaded and the computations starts. You can then stop the execution and run with the `-l` option.Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/82RFW dataset: overlapping and mis-labelling between training and testing sets2022-07-13T13:01:02ZYu LinghuRFW dataset: overlapping and mis-labelling between training and testing setsBased on the datasets we received from Wang et al., when we use z-samples or t-samples as shown below
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/master/bob/bio/face/database/rfw.py#L242, 2 problems occurred during the experiments.
...Based on the datasets we received from Wang et al., when we use z-samples or t-samples as shown below
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/master/bob/bio/face/database/rfw.py#L242, 2 problems occurred during the experiments.
1. There are 44 subjects classified as Caucasian in the training set, but as Indian in the testing set. (e.g. m.0c96fs, m.08y5xt, etc.)
2. When we choose to obtain 2500 z-samples from each race as the cohort, we detect more than 6000 pairs of subjects (one from training and one from testing) that have very high similarity scores (-0.5~-0.1). After manually check some of them, those samples should belong to same person, i.e. not imposter scores. So the overlapping exists between training and testing sets, which is not supposed to be.
This bug report works as a record of problems. I'm not sure if those problems only happen to us because of different versions of datasets.
We could discuss it in a later stage, e.g. use other BUPT datasets like BUPT-Balanced as training set since Wang et al. stated there is no overlap between BUPT-Balanced and RFW, face detection might be necessary since no landmark is given for this dataset.https://gitlab.idiap.ch/bob/bob.devtools/-/issues/101we have to use gcc 102022-06-27T14:26:59ZAmir MOHAMMADIwe have to use gcc 10similar to https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yamlsimilar to https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yamlFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/96Issue with dask distributed and click >=8.1.02022-06-27T14:26:59ZTiago de Freitas PereiraIssue with dask distributed and click >=8.1.0We need to bump our dask, distributed versions because of this issue.
https://github.com/dask/distributed/pull/6018
ping @hotroshiWe need to bump our dask, distributed versions because of this issue.
https://github.com/dask/distributed/pull/6018
ping @hotroshihttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/100The after_script in bob.devtools CI instructions is broken2022-06-27T13:14:02ZAmir MOHAMMADIThe after_script in bob.devtools CI instructions is brokenThis line:
https://gitlab.idiap.ch/bob/bob.devtools/-/blob/dfa4a19a4d2088e9f6e5019ab44c94097d33e78f/.gitlab-ci.yml#L181
Does not work in the CI and silently fails. You can see it in:
https://gitlab.idiap.ch/bob/bob.devtools/-/jobs/27233...This line:
https://gitlab.idiap.ch/bob/bob.devtools/-/blob/dfa4a19a4d2088e9f6e5019ab44c94097d33e78f/.gitlab-ci.yml#L181
Does not work in the CI and silently fails. You can see it in:
https://gitlab.idiap.ch/bob/bob.devtools/-/jobs/272337#L675
```
$ bdt ci clean -vv
/usr/bin/bash: line 129: bdt: command not found
```Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/98The DOCSERVER env variable is no longer used.2022-06-24T08:30:27ZAmir MOHAMMADIThe DOCSERVER env variable is no longer used.I can't remember where this was used. Maybe when using `bob_dbmanage.py download all`? This env variable is not used anywhere anymore and it can be removed from here. Please correct if i am wrong @andre.anjosI can't remember where this was used. Maybe when using `bob_dbmanage.py download all`? This env variable is not used anywhere anymore and it can be removed from here. Please correct if i am wrong @andre.anjosFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/99PYTEST_ADDOPTS does not need to be in recipe_append.yaml2022-06-21T09:18:19ZAmir MOHAMMADIPYTEST_ADDOPTS does not need to be in recipe_append.yamlConda-build only ignores env variables during the build phase, but does not do this at test phase.
Hence, adding `PYTEST_ADDOPTS` and the deprecated `NOSE_EVAL_ATTR` to `recipe_append.yaml` is not needed.Conda-build only ignores env variables during the build phase, but does not do this at test phase.
Hence, adding `PYTEST_ADDOPTS` and the deprecated `NOSE_EVAL_ATTR` to `recipe_append.yaml` is not needed.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.extension/-/issues/88`search_file` scans the whole underlying folder structure2022-06-21T09:06:56ZChristophe ECABERT`search_file` scans the whole underlying folder structureWhen looking into a directory for a file with `search_file(base_path, options)`, the whole structure of the `base_path` get scanned [L395](https://gitlab.idiap.ch/bob/bob.extension/-/blob/master/bob/extension/download.py#L395). The time ...When looking into a directory for a file with `search_file(base_path, options)`, the whole structure of the `base_path` get scanned [L395](https://gitlab.idiap.ch/bob/bob.extension/-/blob/master/bob/extension/download.py#L395). The time complexity increases linearly with the number of underlying files.
This behavior is fine if the intended design was for `options` to be an incomplete relative path within the `base_path`. However if it is expected that `options` are complete relative path, one can speed up the process as follow:
```python
f = None
for o in options:
try:
f = open(os.path.join(base_path, o)
break
except OSError:
pass # File does not exist
return f
```
It implements the same behavior, assuming `options` is the full relative path, and speed up the search.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob/-/issues/273Issues to take care of before release2022-06-10T11:33:30ZAmir MOHAMMADIIssues to take care of before release- [ ] at the end of many pages e.g. https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.bio.spear/README.html there is a contact section with "For questions or reporting issues to this software package, contact our development...- [ ] at the end of many pages e.g. https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.bio.spear/README.html there is a contact section with "For questions or reporting issues to this software package, contact our development mailing list." pointing to https://groups.google.com/g/bob-devel -- isn't it a bit misleading since we are asking people to ask questions on stackoverflow https://stackoverflow.com/questions/tagged/python-bob ?
- [ ] bob.extension is still called a C++ extension package and the documentation needs to be revisited. Most of the useful docs for development should move to bob/bob.devtools
- [x] The tutorial in bob/docs is gone and we need a new one. The other pages need to be revised.
- [ ] bob/bob.bio.base documentation on creating csv datasets is not complete
- [ ] bob/bob.bio.base documentation on legacy should be removed
- [ ] bob/bob.bio.spear documentation and leaderboard needs updating
- [ ] bob/bob.bio.face leaderboard needs updating
- [ ] bob/bob.pad.face needs new baselines. the package is empty as of now.
- [ ] bob/bob.pad.face needs new documentation after baselines have changed.
- [x] bob/bob.bio.vein still uses nosetests instead of pytest
- [ ] bob/bob.devtools we need to command to install all packages inside `bob-devel`. That is creating a conda env with all bob dependencies and without bob packages.
- [x] Check the entry points of all packages (check which ones).The Great Deprecationhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/48VideoPadSample's output is not consistent2022-06-09T12:54:04ZChristophe ECABERTVideoPadSample's output is not consistentThe samples generated by the `bob.pad.face.database.VideoPadSample` are not consistent. The attributes `.data` and `.annotations` do not have the same first dimensions. This behaviour can be reproduce with:
```python
database = ReplayAt...The samples generated by the `bob.pad.face.database.VideoPadSample` are not consistent. The attributes `.data` and `.annotations` do not have the same first dimensions. This behaviour can be reproduce with:
```python
database = ReplayAttackPadDatabase(protocol='smalltest')
samples = database.fit_samples()
vid0 = samples[0].data # vid0.shape == (20, 3, 240, 320)
ann0 = samples[0].annotations # len(ann0) == 375
```
The `.data` attribute contains the sampled frames (i.e. by `VideoAsArray` container) whereas the `.annotations` attribute contains the annotations for the whole video (i.e. they are not sampled)https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/81MTCNN is not serializable2022-06-07T12:10:09ZChristophe ECABERTMTCNN is not serializableWith the current implementation, MTCNN is not serializable via pickle. When used locally, it is not an issue, however, when running on the cluster, we are getting lovely messages into workers' logs such as:
```
distributed.protocol.pick...With the current implementation, MTCNN is not serializable via pickle. When used locally, it is not an issue, however, when running on the cluster, we are getting lovely messages into workers' logs such as:
```
distributed.protocol.pickle - INFO - Failed to serialize CheckpointWrapper(estimator=SampleWrapper(estimator=BoundingBoxAnnotatorCrop(annotator=MTCNN(thresholds=(0.1,
0.2,
0.2)),
eyes_cropper=FaceEyesNorm(final_image_size=(112,
112),
reference_eyes_location={'bottomright': (112,
112),
'leye': (55,
72),
'reye': (55,
40),
'topleft': (0,
0)})),
fit_extra_arguments=(),
input_attribute='data',
output_attribute='data',
transform_...',
'annotations'),)),
extension='.h5',
features_dir='/idiap/temp/cecabert/experiments/bob101/results-sge/ijbc/cropper',
hash_fn=<function hash_string at 0x7fa50d891000>,
load_func=<function load at 0x7fa507f5ed40>,
model_path='/idiap/temp/cecabert/experiments/bob101/results-sge/ijbc/cropper.pkl',
sample_attribute='data',
save_func=<function save at 0x7fa507f5ef80>). Exception: cannot pickle '_thread.RLock' object
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/core.py", line 111, in loads
return msgpack.loads(
File "msgpack/_unpacker.pyx", line 194, in msgpack._cmsgpack.unpackb
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/core.py", line 103, in _decode_default
return merge_and_deserialize(
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/serialize.py", line 488, in merge_and_deserialize
return deserialize(header, merged_frames, deserializers=deserializers)
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/serialize.py", line 417, in deserialize
return loads(header, frames)
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/serialize.py", line 180, in serialization_error_loads
raise TypeError(msg)
TypeError: Could not serialize object of type CheckpointWrapper.
Traceback (most recent call last):
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/pickle.py", line 40, in dumps
result = pickle.dumps(x, **dump_kwargs)
AttributeError: Can't pickle local object 'WeakSet.__init__.<locals>._remove'
```
The issue it that in some case, it tries to serialize the already loaded underlying Tensorflow Graph. This can be solved with the same mechanism used in `PyTorchModel` class by overriding the `__getstate__` method as follow:
```python
def __getstate__(self):
# Handling unpicklable objects
state = {}
for key, value in super().__getstate__().items():
if key != '_fun':
state[key] = value
state['_fun'] = None
return state
```
With this change, the serialization now works properly and can be tested with:
```python
mtcnn = MTCNN()
mtcnn.mtcnn_fun # Force instantiation of TF graph
other = pickle.loads(pickle.dumps(mtcnn))
# No AttributeError: Can't pickle local object 'WeakSet.__init__.<locals>._remove'
# TF graph will be lazily initialized if needed
```https://gitlab.idiap.ch/bob/bob.pad.face/-/issues/45Quality measure Python implementation2022-06-03T15:09:41ZAmir MOHAMMADIQuality measure Python implementationAlso code needs to be re-written to run fast.Also code needs to be re-written to run fast.The Great Deprecationhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/40Could move video processing tools to bob.bio.video2022-06-03T15:09:12ZLaurent COLBOISCould move video processing tools to bob.bio.videoThere are some very nice tools for video processing such as
`bob.pad.face.transformer.VideoToFrames`
or
`bob.pad.face.database.database.delayed_video_load` which are not PAD specific and would gain in visibility if moved to `bob.bio.video`.There are some very nice tools for video processing such as
`bob.pad.face.transformer.VideoToFrames`
or
`bob.pad.face.database.database.delayed_video_load` which are not PAD specific and would gain in visibility if moved to `bob.bio.video`.Laurent COLBOISLaurent COLBOIShttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/30No use of existing annotations in FaceCropAlign2022-06-03T15:08:51ZGuillaume HEUSCHNo use of existing annotations in FaceCropAlignIt looks like the FaceCropAlign class does not take advantage of existing annotations, and hence, face detection is ran no matter what ...It looks like the FaceCropAlign class does not take advantage of existing annotations, and hence, face detection is ran no matter what ...Guillaume HEUSCHGuillaume HEUSCHhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/29Purpose in MaskAttackPadDatabase2022-06-03T15:08:29ZGuillaume HEUSCHPurpose in MaskAttackPadDatabasethe purposes are not handled properly ... considered as a string instead of a listthe purposes are not handled properly ... considered as a string instead of a listGuillaume HEUSCHGuillaume HEUSCHhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/28Unnecessary logger warning?2022-06-03T15:08:14ZAnjith GEORGEanjith.george@idiap.chUnnecessary logger warning?@heusch Do you actually need this warning?
`https://gitlab.idiap.ch/bob/bob.pad.face/blob/master/bob/pad/face/preprocessor/FaceCropAlign.py#L627`
It pops up every time when I have RGB image as input and I want grayscale as the output.@heusch Do you actually need this warning?
`https://gitlab.idiap.ch/bob/bob.pad.face/blob/master/bob/pad/face/preprocessor/FaceCropAlign.py#L627`
It pops up every time when I have RGB image as input and I want grayscale as the output.Guillaume HEUSCHGuillaume HEUSCHhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/25implement loading of annotations for 3DMAD database2022-06-03T15:07:57ZGuillaume HEUSCHimplement loading of annotations for 3DMAD databaseGuillaume HEUSCHGuillaume HEUSCH