bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2022-06-24T08:30:27Zhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/98The DOCSERVER env variable is no longer used.2022-06-24T08:30:27ZAmir MOHAMMADIThe DOCSERVER env variable is no longer used.I can't remember where this was used. Maybe when using `bob_dbmanage.py download all`? This env variable is not used anywhere anymore and it can be removed from here. Please correct if i am wrong @andre.anjosI can't remember where this was used. Maybe when using `bob_dbmanage.py download all`? This env variable is not used anywhere anymore and it can be removed from here. Please correct if i am wrong @andre.anjosFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.extension/-/issues/88`search_file` scans the whole underlying folder structure2022-06-21T09:06:56ZChristophe ECABERT`search_file` scans the whole underlying folder structureWhen looking into a directory for a file with `search_file(base_path, options)`, the whole structure of the `base_path` get scanned [L395](https://gitlab.idiap.ch/bob/bob.extension/-/blob/master/bob/extension/download.py#L395). The time ...When looking into a directory for a file with `search_file(base_path, options)`, the whole structure of the `base_path` get scanned [L395](https://gitlab.idiap.ch/bob/bob.extension/-/blob/master/bob/extension/download.py#L395). The time complexity increases linearly with the number of underlying files.
This behavior is fine if the intended design was for `options` to be an incomplete relative path within the `base_path`. However if it is expected that `options` are complete relative path, one can speed up the process as follow:
```python
f = None
for o in options:
try:
f = open(os.path.join(base_path, o)
break
except OSError:
pass # File does not exist
return f
```
It implements the same behavior, assuming `options` is the full relative path, and speed up the search.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/81MTCNN is not serializable2022-06-07T12:10:09ZChristophe ECABERTMTCNN is not serializableWith the current implementation, MTCNN is not serializable via pickle. When used locally, it is not an issue, however, when running on the cluster, we are getting lovely messages into workers' logs such as:
```
distributed.protocol.pick...With the current implementation, MTCNN is not serializable via pickle. When used locally, it is not an issue, however, when running on the cluster, we are getting lovely messages into workers' logs such as:
```
distributed.protocol.pickle - INFO - Failed to serialize CheckpointWrapper(estimator=SampleWrapper(estimator=BoundingBoxAnnotatorCrop(annotator=MTCNN(thresholds=(0.1,
0.2,
0.2)),
eyes_cropper=FaceEyesNorm(final_image_size=(112,
112),
reference_eyes_location={'bottomright': (112,
112),
'leye': (55,
72),
'reye': (55,
40),
'topleft': (0,
0)})),
fit_extra_arguments=(),
input_attribute='data',
output_attribute='data',
transform_...',
'annotations'),)),
extension='.h5',
features_dir='/idiap/temp/cecabert/experiments/bob101/results-sge/ijbc/cropper',
hash_fn=<function hash_string at 0x7fa50d891000>,
load_func=<function load at 0x7fa507f5ed40>,
model_path='/idiap/temp/cecabert/experiments/bob101/results-sge/ijbc/cropper.pkl',
sample_attribute='data',
save_func=<function save at 0x7fa507f5ef80>). Exception: cannot pickle '_thread.RLock' object
distributed.protocol.core - CRITICAL - Failed to deserialize
Traceback (most recent call last):
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/core.py", line 111, in loads
return msgpack.loads(
File "msgpack/_unpacker.pyx", line 194, in msgpack._cmsgpack.unpackb
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/core.py", line 103, in _decode_default
return merge_and_deserialize(
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/serialize.py", line 488, in merge_and_deserialize
return deserialize(header, merged_frames, deserializers=deserializers)
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/serialize.py", line 417, in deserialize
return loads(header, frames)
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/serialize.py", line 180, in serialization_error_loads
raise TypeError(msg)
TypeError: Could not serialize object of type CheckpointWrapper.
Traceback (most recent call last):
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/distributed/protocol/pickle.py", line 40, in dumps
result = pickle.dumps(x, **dump_kwargs)
AttributeError: Can't pickle local object 'WeakSet.__init__.<locals>._remove'
```
The issue it that in some case, it tries to serialize the already loaded underlying Tensorflow Graph. This can be solved with the same mechanism used in `PyTorchModel` class by overriding the `__getstate__` method as follow:
```python
def __getstate__(self):
# Handling unpicklable objects
state = {}
for key, value in super().__getstate__().items():
if key != '_fun':
state[key] = value
state['_fun'] = None
return state
```
With this change, the serialization now works properly and can be tested with:
```python
mtcnn = MTCNN()
mtcnn.mtcnn_fun # Force instantiation of TF graph
other = pickle.loads(pickle.dumps(mtcnn))
# No AttributeError: Can't pickle local object 'WeakSet.__init__.<locals>._remove'
# TF graph will be lazily initialized if needed
```https://gitlab.idiap.ch/bob/nightlies/-/issues/63Not all bob packages are taken from local channel2022-06-01T09:44:27ZAmir MOHAMMADINot all bob packages are taken from local channelJob [#270279](https://gitlab.idiap.ch/bob/nightlies/-/jobs/270279) failed for 526f22f2ac40ed01c6b54ffcb992ef6eeeeefab5:
Somehow, mamba is not picking up all bob packages from the local conda channel:
```
bob.bio.base: ...Job [#270279](https://gitlab.idiap.ch/bob/nightlies/-/jobs/270279) failed for 526f22f2ac40ed01c6b54ffcb992ef6eeeeefab5:
Somehow, mamba is not picking up all bob packages from the local conda channel:
```
bob.bio.base: 6.0.1b0-py38h44fe14c_25 local
bob.bio.face: 6.0.1b0-py38h4d93ce0_18 local
bob.bio.video: 5.0.1b0-py38hd5d7b21_13 local
bob.extension: 6.1.2b0-py38h460ab40_32 local
bob.io.base: 4.0.1b0-py38h2539f08_35 local
bob.learn.em: 3.0.0b0-py38h4769f04_18 http://www.idiap.ch/software/bob/conda/label/beta
bob.measure: 5.0.1b0-py38h521d9d0_31 local
bob.pipelines: 2.0.1b0-py38hbfafce2_17 http://www.idiap.ch/software/bob/conda/label/beta
```
I don't know why yet but that is why the nightlies are broken.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/97Doc issues2022-05-31T13:21:29ZLaurent COLBOISDoc issues### 1. `pre-commit` alias doesn't work properly
##### Run
```
bdt dev checkout bob.bio.base
```
##### Error
```
Traceback (most recent call last):
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/bin/bdt", line 11, in <module>
sys.e...### 1. `pre-commit` alias doesn't work properly
##### Run
```
bdt dev checkout bob.bio.base
```
##### Error
```
Traceback (most recent call last):
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/bin/bdt", line 11, in <module>
sys.exit(main())
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/bob/devtools/scripts/development.py", line 70, in checkout
subprocess.check_call(["pre-commit", "install"], cwd=dest)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/subprocess.py", line 368, in check_call
retcode = call(*popenargs, **kwargs)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/subprocess.py", line 349, in call
with Popen(*popenargs, **kwargs) as p:
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/subprocess.py", line 951, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/subprocess.py", line 1821, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'pre-commit'
```
### 2. Additional information required regarding virtual packages (I think)
##### Checkout `bob.bio.face` and run :
```
bdt dev create -vv dev
```
##### Error:
```
The reported errors are:
- Encountered problems while solving:
- - nothing provides __cuda needed by tensorflow-2.7.0-cuda102py310hcf4adbc_0
-
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/bin/bdt", line 11, in <module>
sys.exit(main())
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/bob/devtools/scripts/bdt.py", line 43, in _decorator
value = view_func(*args, **kwargs)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/bob/devtools/scripts/create.py", line 268, in create
deps = parse_dependencies(recipe_dir, conda_config)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/bob/devtools/build.py", line 333, in parse_dependencies
metadata = get_rendered_metadata(recipe_dir, config)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/bob/devtools/build.py", line 239, in get_rendered_metadata
return conda_build.api.render(recipe_dir, config=config)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/api.py", line 42, in render
metadata_tuples = render_recipe(recipe_path, bypass_env_check=bypass_env_check,
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/render.py", line 860, in render_recipe
rendered_metadata = distribute_variants(m, variants,
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/render.py", line 763, in distribute_variants
mv.parse_until_resolved(allow_no_other_outputs=allow_no_other_outputs,
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/metadata.py", line 1084, in parse_until_resolved
self.parse_again(permit_undefined_jinja=False,
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/metadata.py", line 1006, in parse_again
self.meta = parse(self._get_contents(permit_undefined_jinja,
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/metadata.py", line 1601, in _get_contents
rendered = template.render(environment=env)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "/remote/idiap.svm/user.active/lcolbois/bob_dev/run_ijbc/bob.bio.face/conda/meta.yaml", line 41, in top-level template code
- setuptools
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/jinja_context.py", line 228, in pin_compatible
pins, _, _ = get_env_dependencies(m, 'host', m.config.variant)
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/conda_build/render.py", line 138, in get_env_dependencies
actions = environ.get_install_actions(tmpdir, tuple(dependencies), env,
File "/idiap/temp/lcolbois/miniconda3/envs/bdt/lib/python3.9/site-packages/boa/cli/mambabuild.py", line 133, in mamba_get_install_actions
raise err
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {MatchSpec("tensorflow==2.7.0=cuda102py310hcf4adbc_0")}
```
##### Possible fix : first run
```
export CONDA_OVERRIDE_CUDA=11.6
```Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/185Probe template not accepted if not numpy array2022-05-31T08:57:06ZYannick DAYERProbe template not accepted if not numpy arrayIn `score_sample_template` from `abstract_classes.py` there is a check that the data is "valid" ([here](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/bob/bio/base/pipelines/abstract_classes.py#L246)).
It checks that the templat...In `score_sample_template` from `abstract_classes.py` there is a check that the data is "valid" ([here](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/master/bob/bio/base/pipelines/abstract_classes.py#L246)).
It checks that the template in the samples is a Numpy array and is not empty. But in GMM for example, the templates are objects (`GMMStats`) and all the samples are considered "invalid" and ignored silently.
We could either remove the check or implement a way of detecting non-Numpy objects and allow those unconditionally.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.spear/-/issues/42ImportError: cannot import name 'SpeechbrainEmbeddings' from 'bob.bio.spear.e...2022-05-25T15:00:39ZHatef OTROSHIImportError: cannot import name 'SpeechbrainEmbeddings' from 'bob.bio.spear.extractor'In the master branch, in [bob/bio/spear/config/pipeline/speechbrain_ecapa_voxceleb.py#L4](https://gitlab.idiap.ch/bob/bob.bio.spear/-/blob/ddf128c6da2872d7ec5a2608b4fddf03b3884537/bob/bio/spear/config/pipeline/speechbrain_ecapa_voxceleb....In the master branch, in [bob/bio/spear/config/pipeline/speechbrain_ecapa_voxceleb.py#L4](https://gitlab.idiap.ch/bob/bob.bio.spear/-/blob/ddf128c6da2872d7ec5a2608b4fddf03b3884537/bob/bio/spear/config/pipeline/speechbrain_ecapa_voxceleb.py#L4) there is `from bob.bio.spear.extractor import SpeechbrainEmbeddings` but `SpeechbrainEmbeddings` cannot be imported.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/184cannot import 'isinstance_nested' from 'bob.pipelines.utils'2022-05-24T17:23:17ZHatef OTROSHIcannot import 'isinstance_nested' from 'bob.pipelines.utils'In the master branch, in [bob/bio/base/pipelines/pipelines.py#L18](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/1abaa5f11559c15ee185203a65f8480800524b01/bob/bio/base/pipelines/pipelines.py#L18) there is `from bob.pipelines.utils impor...In the master branch, in [bob/bio/base/pipelines/pipelines.py#L18](https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/1abaa5f11559c15ee185203a65f8480800524b01/bob/bio/base/pipelines/pipelines.py#L18) there is `from bob.pipelines.utils import isinstance_nested` but `isinstance_nested` is not available in master branch of [bob/pipelines/utils.py](https://gitlab.idiap.ch/bob/bob.pipelines/-/blob/611f8867a16711bf0394cc9af2901bd4941b6a5b/bob/pipelines/utils.py)https://gitlab.idiap.ch/bob/bob.pad.face/-/issues/48VideoPadSample's output is not consistent2022-06-09T12:54:04ZChristophe ECABERTVideoPadSample's output is not consistentThe samples generated by the `bob.pad.face.database.VideoPadSample` are not consistent. The attributes `.data` and `.annotations` do not have the same first dimensions. This behaviour can be reproduce with:
```python
database = ReplayAt...The samples generated by the `bob.pad.face.database.VideoPadSample` are not consistent. The attributes `.data` and `.annotations` do not have the same first dimensions. This behaviour can be reproduce with:
```python
database = ReplayAttackPadDatabase(protocol='smalltest')
samples = database.fit_samples()
vid0 = samples[0].data # vid0.shape == (20, 3, 240, 320)
ann0 = samples[0].annotations # len(ann0) == 375
```
The `.data` attribute contains the sampled frames (i.e. by `VideoAsArray` container) whereas the `.annotations` attribute contains the annotations for the whole video (i.e. they are not sampled)https://gitlab.idiap.ch/bob/bob.pad.face/-/issues/47Vanilla-pad baseline does not run2022-05-31T08:57:22ZChristophe ECABERTVanilla-pad baseline does not runFollowing the [documentation](https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.pad.base/doc/vanilla_pad_intro.html#running-a-biometric-experiment-with-vanilla-pad), the vanilla-pad baseline can be invoked with:
```python
b...Following the [documentation](https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.pad.base/doc/vanilla_pad_intro.html#running-a-biometric-experiment-with-vanilla-pad), the vanilla-pad baseline can be invoked with:
```python
bob pad vanilla-pad replay-attack svm-frames -o results -vv
```
This leads to the following exception:
```python
Traceback (most recent call last):
File "/remote/idiap.svm/temp.biometric03/cecabert/bob_beta/src/bob.pipelines/bob/pipelines/wrappers.py", line 808, in _fit
self.estimator = self.estimator.fit(X, y, **fit_params)
File "/remote/idiap.svm/temp.biometric03/cecabert/bob_beta/src/bob.pipelines/bob/pipelines/wrappers.py", line 337, in fit
self.estimator = self.estimator.fit(X, **kwargs)
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/sklearn/svm/_base.py", line 190, in fit
X, y = self._validate_data(
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/sklearn/base.py", line 581, in _validate_data
X, y = check_X_y(X, y, **check_params)
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/sklearn/utils/validation.py", line 964, in check_X_y
X = check_array(
File "/remote/idiap.svm/temp.biometric03/cecabert/mambaforge/envs/bob_deps/lib/python3.10/site-packages/sklearn/utils/validation.py", line 794, in check_array
raise ValueError(
ValueError: Found array with dim 4. Estimator expected <= 2.
```
The `fit` method of the SVM classifier is expecting its input to be `(n_samples, n_features)` which is not what is provided by the `VideoToFrames` transformer. One possible solution would be to add an intermediate step between the two operations to flatten the data.https://gitlab.idiap.ch/bob/bob.pipelines/-/issues/41Expanding samples on the fly2022-05-31T13:53:53ZAnjith GEORGEanjith.george@idiap.chExpanding samples on the flyI have a usecase where I want to expand one image to multiple images in the `pipelinesimple`. Essentially, I can create
a transformer which can operate directly on samples, which takes in a `SampleSet` with one image and return a `Sampl...I have a usecase where I want to expand one image to multiple images in the `pipelinesimple`. Essentially, I can create
a transformer which can operate directly on samples, which takes in a `SampleSet` with one image and return a `SampleSet` with `n` samples.
I managed to make it work, when every thing is in memory (with `-m` option), and checkpointing is a problem since there are new samples. I can create new keys on the fly, what do you think is the best way to go about this.
@amohammadi @tiago.pereira @ydayerhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/183Heterogeneous pipelines for identification2022-05-31T13:54:33ZAnjith GEORGEanjith.george@idiap.chHeterogeneous pipelines for identificationI am trying to run a face recognition pipline where the pipeline for enrollment and probes are different, basically my enrollment part has one step extra. Currently I have created a new heterogeneous pipeline which takes in two different...I am trying to run a face recognition pipline where the pipeline for enrollment and probes are different, basically my enrollment part has one step extra. Currently I have created a new heterogeneous pipeline which takes in two different pipelines as the inputs. I think its not very efficient, since there are common components in both of the pipelines (Iresnet in my case). I think it would be cleaner, if there is a transform which can do things conditionally, meaning, if it sees that the sample is from the enrollment set then apply the transform and skup it if its not from enrollment. Adding the group attribute at the csvloader level might solve this, so that I can do the filtering in this transform.
Any suggestions welcome.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/182Unable to run the pipeline after the last update of bob.bio.base2022-05-03T16:26:00ZYu LinghuUnable to run the pipeline after the last update of bob.bio.baseThe latest version of bob.bio.base does not support to run any `simple` or `score-norm` pipeline. Below is the example error message:
```
▶ ./bin/bob bio pipeline simple rfw-idiap arcface-insightface -vvv -o idiap_arcface
Warning: entr...The latest version of bob.bio.base does not support to run any `simple` or `score-norm` pipeline. Below is the example error message:
```
▶ ./bin/bob bio pipeline simple rfw-idiap arcface-insightface -vvv -o idiap_arcface
Warning: entry point could not be loaded. Contact its author for help.
Traceback (most recent call last):
File "/local/scratch/linghu/miniconda3/envs/bob.paper.frice2/lib/python3.8/site-packages/click_plugins/core.py", line 37, in decorator
group.add_command(entry_point.load())
File "/local/scratch/linghu/miniconda3/envs/bob.paper.frice2/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/local/scratch/linghu/miniconda3/envs/bob.paper.frice2/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2456, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/__init__.py", line 1, in <module>
from . import ( # noqa: F401
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/algorithm/__init__.py", line 1, in <module>
from .Algorithm import Algorithm
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/algorithm/Algorithm.py", line 10, in <module>
from .. import utils
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/utils/__init__.py", line 12, in <module>
from .io import * # noqa: F401,F403
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/utils/io.py", line 13, in <module>
from bob.bio.base import database
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/database/__init__.py", line 1, in <module>
from . import filelist # noqa: F401
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/database/filelist/__init__.py", line 2, in <module>
from .query import FileListBioDatabase
File "/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/database/filelist/query.py", line 10, in <module>
from .. import BioFile, ZTBioDatabase
ImportError: cannot import name 'BioFile' from partially initialized module 'bob.bio.base.database' (most likely due to a circular import) (/local/scratch/linghu/bob.bio.face/src/bob.bio.base/bob/bio/base/database/__init__.py)
```https://gitlab.idiap.ch/bob/bob.pad.face/-/issues/46Nightlies failing because of this one2022-05-02T18:15:24ZTiago de Freitas PereiraNightlies failing because of this one
https://gitlab.idiap.ch/bob/nightlies/-/jobs/266637
```
======================================================================
FAIL: Failure: AssertionError (File: not found: /Users/gitlab/builds/6eE8C-FW/1/bob/nightlies/miniconda/co...
https://gitlab.idiap.ch/bob/nightlies/-/jobs/266637
```
======================================================================
FAIL: Failure: AssertionError (File: not found: /Users/gitlab/builds/6eE8C-FW/1/bob/nightlies/miniconda/conda-bld/bob.pad.face_1651233096650/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.9/site-packages/bob/pad/face/test/data/real_client001_android_SD_scene01.mp4)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/gitlab/builds/6eE8C-FW/1/bob/nightlies/miniconda/conda-bld/bob.pad.face_1651233096650/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.9/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
```
I think it's just an issue with the MANIFEST.https://gitlab.idiap.ch/bob/bob.learn.em/-/issues/45ISV dask implementation is not efficient and workers run out of memory2022-05-09T13:00:47ZAmir MOHAMMADIISV dask implementation is not efficient and workers run out of memoryThe Great DeprecationAmir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/45Quality measure Python implementation2022-06-03T15:09:41ZAmir MOHAMMADIQuality measure Python implementationAlso code needs to be re-written to run fast.Also code needs to be re-written to run fast.The Great Deprecationhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/80Cannot specify original directory and extension for most of the databases any...2022-05-25T09:16:16ZManuel Günthersiebenkopf@googlemail.comCannot specify original directory and extension for most of the databases anymoreWhile in previous database implementations, it was relatively straightforward to utilize the database interface in order to load pre-extracted features, in the new database interfaces this is no longer possible, for two reasons:
1. Ther...While in previous database implementations, it was relatively straightforward to utilize the database interface in order to load pre-extracted features, in the new database interfaces this is no longer possible, for two reasons:
1. There is no simple way of providing the database interface with the directory where to read the data from. For example, in an old interface (LFW), we can still set:https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/c7ee7213f83f62b1e36685290e1defd10fea2c20/bob/bio/face/database/lfw.py#L90, while this option does no longer exist in newer interfaces: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/c7ee7213f83f62b1e36685290e1defd10fea2c20/bob/bio/face/database/scface.py#L30, although it should be relatively straightforward to implement that since the default value is used just a few lines below: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/c7ee7213f83f62b1e36685290e1defd10fea2c20/bob/bio/face/database/scface.py#L47. It should be simple to re-expose this option to the user.
2. The filename extension is by default empty: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/c7ee7213f83f62b1e36685290e1defd10fea2c20/bob/bio/face/database/scface.py#L50, and there is no possibility to change that on the constructor. But even if we would expose the extension similarly to the directory name, we would still be in trouble since the default loader just **appends** the extension rather than ** replacing** it: https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/406f2da1faadacd4d4fe4e36e5a0010d78557513/bob/bio/base/database/csv_dataset.py#L145
The main issue with 2. is that someone has decided to ignore our old behavior to store keys without filename extension, and just added the original extension to the key. For example, running:
```
import bob.bio.face
db = bob.bio.face.database.SCFaceDatabase(protocol="far")
samples = db.all_samples()
print(samples[0].key)
```
will print `filename.JPG` instead of `filename` (without extension).
So, what I would propose is (the least amount of changes, anything else would require to re-create all the CSV-based databases):
- [ ] Expose the `original_directory` and `original_extension` parameters for all databases to the user, keeping their default values as they currently are.
- [ ] Change the line in https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/406f2da1faadacd4d4fe4e36e5a0010d78557513/bob/bio/base/database/csv_dataset.py#L145 to replace the extension with the given one (if one is given) rather than appending it to the filename.
Any objection?https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/180It is not possible to run experiments without dask wrapper2022-04-26T12:06:41ZAmir MOHAMMADIIt is not possible to run experiments without dask wrapperNot using dask is useful especially when you want to debug your code. With the current implementation, this is not possible.Not using dask is useful especially when you want to debug your code. With the current implementation, this is not possible.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/nightlies/-/issues/61bob.ip.binseg Tests hang, Job Failed #2655822022-05-12T13:37:25ZAmir MOHAMMADIbob.ip.binseg Tests hang, Job Failed #265582Job [#265582](https://gitlab.idiap.ch/bob/nightlies/-/jobs/265582) failed for e03c39bce11c74e60373f99d6098d3bf7f75c4b6:
Looking at macOS builds, most probably the hanging test is:
```
ip/binseg/test/test_cli.py::test_experiment_stare_wi...Job [#265582](https://gitlab.idiap.ch/bob/nightlies/-/jobs/265582) failed for e03c39bce11c74e60373f99d6098d3bf7f75c4b6:
Looking at macOS builds, most probably the hanging test is:
```
ip/binseg/test/test_cli.py::test_experiment_stare_with_multiprocessing PASSED [ 58%]
```André AnjosAndré Anjoshttps://gitlab.idiap.ch/bob/bob.pad.base/-/issues/43The documentation of the file list database is outdated2022-05-04T13:26:04ZTiago de Freitas PereiraThe documentation of the file list database is outdatedDuring the workshop, we spotted the documentation of the file list database interface (https://www.idiap.ch/software/bob/docs/bob/bob.pad.base/master/vanilla_pad_features.html) is outdated.
It doesn't match with what is currently implem...During the workshop, we spotted the documentation of the file list database interface (https://www.idiap.ch/software/bob/docs/bob/bob.pad.base/master/vanilla_pad_features.html) is outdated.
It doesn't match with what is currently implemented.
ping @ydayer @amohammadiYannick DAYERYannick DAYER