bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2022-03-18T09:09:57Zhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/143Adding tags to transformers to differentiate between annotators, preprocessor...2022-03-18T09:09:57ZAmir MOHAMMADIAdding tags to transformers to differentiate between annotators, preprocessors, and extractors.I think while it was a good idea to use scikit-learn's transformers API for our classes, still there are some differences between the transformers that we implement. I suggest adding tags to our transformers (https://scikit-learn.org/sta...I think while it was a good idea to use scikit-learn's transformers API for our classes, still there are some differences between the transformers that we implement. I suggest adding tags to our transformers (https://scikit-learn.org/stable/developers/develop.html#estimator-tags) to be able to programmatically differentiate them.
For example, we can have:
```python
class Preprocessor(BaseEstimator):
def _more_tags(self):
return {'bob_transformer': 'preprocessor'}
```
that would allow:
```python
preprocessor = wrap(["sample"], preprocessor)
```
to implicitly imply:
```python
transform_extra_arguments = (("annotations", "annotations"),)
preprocessor = wrap(["sample"], preprocessor, transform_extra_arguments=transform_extra_arguments)
```
Or wrapping an annotator would imply `sample.annotations = annotator(sample.data)` instead of the usual `sample.data = transformer(sample.data)`.
What do you think? Does it make sense?Bob 9.0.0Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.io.audio/-/issues/7bob.io.matlab is deprecated2020-11-02T17:13:09ZAmir MOHAMMADIbob.io.matlab is deprecatedbob.io.matlab usage needs to be dropped here.
see https://gitlab.idiap.ch/bob/bob.io.matlab/-/issues/5
```
bob.io.audio/requirements.txt
7:bob.io.matlab
bob.io.audio/bob/io/audio/test.py
14:import bob.io.matlab
bob.io.audio/develop.cfg...bob.io.matlab usage needs to be dropped here.
see https://gitlab.idiap.ch/bob/bob.io.matlab/-/issues/5
```
bob.io.audio/requirements.txt
7:bob.io.matlab
bob.io.audio/bob/io/audio/test.py
14:import bob.io.matlab
bob.io.audio/develop.cfg
15: src/bob.io.matlab
28:bob.io.matlab = git https://gitlab.idiap.ch/bob/bob.io.matlab
bob.io.audio/conda/meta.yaml
32: - bob.io.matlab
```Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/23Drop dependency on bob.io.matlab2020-11-03T08:31:33ZAmir MOHAMMADIDrop dependency on bob.io.matlabLooks like bob.io.matlab is listed here but not used:
```
bob.bio.vein/requirements.txt
12:bob.io.matlab
bob.bio.vein/bob/bio/vein/tests/test.py
22:import bob.io.matlab
bob.bio.vein/develop.cfg
22: src/bob.io.matlab
52:bob.io....Looks like bob.io.matlab is listed here but not used:
```
bob.bio.vein/requirements.txt
12:bob.io.matlab
bob.bio.vein/bob/bio/vein/tests/test.py
22:import bob.io.matlab
bob.bio.vein/develop.cfg
22: src/bob.io.matlab
52:bob.io.matlab = git git@gitlab.idiap.ch:bob/bob.io.matlab
bob.bio.vein/matlab/compare.py
9:import bob.io.matlab
bob.bio.vein/conda/meta.yaml
39: - bob.io.matlab
```
bob.io.matlab is deprecated https://gitlab.idiap.ch/bob/bob.io.matlab/-/issues/5Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/142Decoupling data and annotation loading of BioFile's from database interfaces2022-03-03T17:35:48ZAmir MOHAMMADIDecoupling data and annotation loading of BioFile's from database interfacesAs originally proposed in https://gitlab.idiap.ch/bob/bob.db.base/-/issues/22
I think it would make sense to separate data loading and annotation loading from the `database` object and put the functionality in `BioFile`s.
This effort wou...As originally proposed in https://gitlab.idiap.ch/bob/bob.db.base/-/issues/22
I think it would make sense to separate data loading and annotation loading from the `database` object and put the functionality in `BioFile`s.
This effort would also be aligned with our idea of samples in bob.pipelines. Currently I have this:
```python
class BioFile(bob.db.base.File, _ReprMixin):
"""
A simple base class that defines basic properties of File object for the use
in verification experiments
Attributes
----------
client_id : str or int
The id of the client this file belongs to.
Its type depends on your implementation.
If you use an SQL database, this should be an SQL type like Integer or
String.
path : object
see :py:class:`bob.db.base.File` constructor
file_id : object
see :py:class:`bob.db.base.File` constructor
original_directory : str or None
The path to the original directory of the file
original_extension : str or None
The extension of the original files. This attribute is deprecated.
Please try to include the extension in the ``path`` attribute
annotation_directory : str or None
The path to the directory of the annotations
annotation_extension : str or None
The extension of annotation files. Default is ``.json``
annotation_type : str or None
The type of the annotation file, see
:any:`bob.db.base.annotations.read_annotation_file`. Default is
``json``.
"""
def __init__(
self,
client_id,
path,
file_id=None,
original_directory=None,
original_extension=None,
annotation_directory=None,
annotation_extension=None,
annotation_type=None,
**kwargs,
):
super(BioFile, self).__init__(path, file_id, **kwargs)
# just copy the information
self.client_id = client_id
"""The id of the client, to which this file belongs to."""
self.original_directory = original_directory
self.original_extension = original_extension
self.annotation_directory = annotation_directory
self.annotation_extension = annotation_extension or ".json"
self.annotation_type = annotation_type or "json"
def load(self):
"""Loads the data at the specified location and using the given extension.
Override it if you need to load differently.
Returns
-------
object
The loaded data (normally :py:class:`numpy.ndarray`).
"""
# get the path
path = self.make_path(
self.original_directory or "", self.original_extension or ""
)
return bob.io.base.load(path)
@property
def annotations(self):
path = self.make_path(self.annotation_directory or "", self.annotation_extension or "")
return read_annotation_file(path, annotation_type=self.annotation_type)
```
which requires a refactoring of our high-level db interfaces (we will not touch low-level db interfaces).
What do you think?
Of course, the load and annotations methods can be overridden per db.Bob 9.0.0Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.extension/-/issues/76do not propogate __ variables when config chain loading2022-03-03T17:39:26ZAmir MOHAMMADIdo not propogate __ variables when config chain loadingCurrently, there is no way to hide variables when chain loading config files and these variables could affect the command behavior or other config files.Currently, there is no way to hide variables when chain loading config files and these variables could affect the command behavior or other config files.Bob 9.0.0Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/138Port annotators to the scikit-learn transformers API2020-12-16T12:01:28ZAmir MOHAMMADIPort annotators to the scikit-learn transformers APIOur annotators are very simple like preprocessors, I think they would also benefit from becoming scikit-learn transformers, particularly allowing batch processing.Our annotators are very simple like preprocessors, I think they would also benefit from becoming scikit-learn transformers, particularly allowing batch processing.Bob 9.0.0https://gitlab.idiap.ch/bob/bob.ip.tensorflow_extractor/-/issues/11Port to tensorflow 22021-03-16T17:22:28ZAmir MOHAMMADIPort to tensorflow 2We need to port this package to tensorflow 2 and remove unused parts.We need to port this package to tensorflow 2 and remove unused parts.Bob 9.0.0https://gitlab.idiap.ch/bob/bob.db.swan/-/issues/4Follow-up from "[bob9] Rewired imports from bob.bio.* to bob.bio.*_legacy tha...2022-05-16T14:42:55ZAmir MOHAMMADIFollow-up from "[bob9] Rewired imports from bob.bio.* to bob.bio.*_legacy that are in bob.pad.base"The following discussion from !5 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/bob/bob.db.swan/-/merge_requests/5#note_56377): (+1 comment)
> wasn't this API was supposed to stay in bob.bio....The following discussion from !5 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/bob/bob.db.swan/-/merge_requests/5#note_56377): (+1 comment)
> wasn't this API was supposed to stay in bob.bio.base?Bob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/137Follow-up from "Correct the chain loading and click implementation"2021-10-29T15:34:56ZAmir MOHAMMADIFollow-up from "Correct the chain loading and click implementation"The following discussion from !201 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/201#note_56304): (+2 comments)
> @tiago.pereira As far as I remember this ...The following discussion from !201 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/201#note_56304): (+2 comments)
> @tiago.pereira As far as I remember this function was not modifying objects in place, was it?Bob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/docs/-/issues/9Follow-up from "[archive] Archiving bob.pad.vein [skip-ci]"2020-12-18T08:28:31ZAmir MOHAMMADIFollow-up from "[archive] Archiving bob.pad.vein [skip-ci]"The following discussion from !19 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/bob/docs/-/merge_requests/19#note_56282):
> I am sure there are more places where bob.pad.vein is referred.The following discussion from !19 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/bob/docs/-/merge_requests/19#note_56282):
> I am sure there are more places where bob.pad.vein is referred.Bob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.pipelines/-/issues/20Problem while using `sge_default` dask client2020-10-12T11:25:50ZVictor BROSProblem while using `sge_default` dask clientFOr some reason `bob.pipelines.distributed.sge.SGEIdiapJob` is requiring a class variable `config_name`.
I've patched myself to make it work, but this needs a proper fix
```
bob@2020-10-09 14:36:37,427 -- DEBUG: Logging of the `bob' log...FOr some reason `bob.pipelines.distributed.sge.SGEIdiapJob` is requiring a class variable `config_name`.
I've patched myself to make it work, but this needs a proper fix
```
bob@2020-10-09 14:36:37,427 -- DEBUG: Logging of the `bob' logger was set to 3
bob.extension.config@2020-10-09 14:36:37,430 -- DEBUG: Loading configuration file `./experiments/vera-finger/veradb.py'...
bob.extension.config@2020-10-09 14:36:38,765 -- DEBUG: Loading configuration file `./experiments/vera-finger/vera_miura.py'...
bob.bio.base@2020-10-09 14:36:39,001 -- INFO: Using `bob.bio.base` legacy algorithm <class 'bob.bio.vein.algorithm.MiuraMatch'>(ch=80, cw=90, multiple_model_scoring='average', multiple_probe_scoring='average')
bob.extension.config@2020-10-09 14:36:39,002 -- DEBUG: Loading configuration file `./src/bob.pipelines/bob/pipelines/config/distributed/sge_default.py'...
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f92cb83db50>>, <Task finished coro=<SpecCluster._correct_state_internal() done, defined at /remote/idiap.svm/temp.biometric01/vbros/bob_vein/bob.bio.vein/eggs/distributed-2.30.0-py3.7.egg/distributed/deploy/spec.py:320> exception=ValueError("The class <class 'bob.pipelines.distributed.sge.SGEIdiapJob'> is required to have a 'config_name' class variable.\nIf you have created this class, please add a 'config_name' class variable.\nIf not this may be a bug, feel free to create an issue at: https://github.com/dask/dask-jobqueue/issues/new")>)
Traceback (most recent call last):
File "/idiap/temp/vbros/miniconda3/envs/bob.bio.vein/lib/python3.7/site-packages/tornado/ioloop.py", line 743, in _run_callback
ret = callback()
File "/idiap/temp/vbros/miniconda3/envs/bob.bio.vein/lib/python3.7/site-packages/tornado/ioloop.py", line 767, in _discard_future_result
future.result()
File "/remote/idiap.svm/temp.biometric01/vbros/bob_vein/bob.bio.vein/eggs/distributed-2.30.0-py3.7.egg/distributed/deploy/spec.py", line 348, in _correct_state_internal
worker = cls(self.scheduler.address, **opts)
File "/remote/idiap.svm/temp.biometric01/vbros/bob_vein/bob.bio.vein/src/bob.pipelines/bob/pipelines/distributed/sge.py", line 56, in __init__
super().__init__(*args, config_name=config_name, death_timeout=10000, **kwargs)
File "/remote/idiap.svm/temp.biometric01/vbros/bob_vein/bob.bio.vein/eggs/dask_jobqueue-0.7.1-py3.7.egg/dask_jobqueue/core.py", line 156, in __init__
default_config_name = self.default_config_name()
File "/remote/idiap.svm/temp.biometric01/vbros/bob_vein/bob.bio.vein/eggs/dask_jobqueue-0.7.1-py3.7.egg/dask_jobqueue/core.py", line 260, in default_config_name
"https://github.com/dask/dask-jobqueue/issues/new".format(cls)
ValueError: The class <class 'bob.pipelines.distributed.sge.SGEIdiapJob'> is required to have a 'config_name' class variable.
```Bob 9.0.0Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.pipelines/-/issues/19Dask Client as python resources2020-10-12T14:19:51ZTiago de Freitas PereiraDask Client as python resourcesWe should put the Dask Clients from here: https://gitlab.idiap.ch/bob/bob.pipelines/-/tree/master/bob/pipelines/config/distributed
as python resources.We should put the Dask Clients from here: https://gitlab.idiap.ch/bob/bob.pipelines/-/tree/master/bob/pipelines/config/distributed
as python resources.Bob 9.0.0Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/37Dask Pipelines2022-04-12T16:37:40ZTiago de Freitas PereiraDask PipelinesEfforts to be done to use `bob.pipelines`Efforts to be done to use `bob.pipelines`Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.video/-/issues/15Port Dask Pipelines2021-09-09T07:00:52ZTiago de Freitas PereiraPort Dask Pipelines* [x] wrappers for sciki-learn estimators
* [x] wrapper for BiometricAlgorithm
* [x] documentation
* [x] Remove FrameContainer, FrameSelector, and wrappers for old preprocessors, extractors, and algorithms
* [x] Port VideoBioFile to use ...* [x] wrappers for sciki-learn estimators
* [x] wrapper for BiometricAlgorithm
* [x] documentation
* [x] Remove FrameContainer, FrameSelector, and wrappers for old preprocessors, extractors, and algorithms
* [x] Port VideoBioFile to use VideoAsArray and VideoLikeContainer.Bob 9.0.0Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/21Dask Pipelines2020-12-03T08:37:30ZTiago de Freitas PereiraDask PipelinesThis package needs to be ported to the new API (https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/180).
The job is extremely easy, though.
I short we need to:
- [x] Wrap all the preprocessors and extractors from here (https://...This package needs to be ported to the new API (https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/180).
The job is extremely easy, though.
I short we need to:
- [x] Wrap all the preprocessors and extractors from here (https://gitlab.idiap.ch/bob/bob.bio.vein/-/tree/master/bob/bio/vein/configurations) to their corresponding transformers https://gitlab.idiap.ch/bob/bob.bio.base/-/tree/dask-pipelines/bob/bio/base/transformers
- [x] Wrap all the algorithms from here https://gitlab.idiap.ch/bob/bob.bio.vein/-/tree/master/bob/bio/vein/configurations using this
https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/dask-pipelines/bob/bio/base/pipelines/vanilla_biometrics/legacy.py#L183
- [ ] Wrap all the databases from here https://gitlab.idiap.ch/bob/bob.bio.vein/-/tree/master/bob/bio/vein/configurations using this https://gitlab.idiap.ch/bob/bob.bio.base/-/blob/dask-pipelines/bob/bio/base/pipelines/vanilla_biometrics/legacy.py#L39
This will get 95% of the task done.
The other 5% are minor adjustments with documentation, extra test cases, and possible bugs that might be risen
Is anyone willing to help?
THanksBob 9.0.0Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.pad.base/-/issues/34TEMPORARY Detach from bob.bio.base2020-10-08T11:42:46ZTiago de Freitas PereiraTEMPORARY Detach from bob.bio.baseHi guys,
We are about to have a beta release of `bob.bio.base` and `bob.bio.face`.
Thanks to @ydayer, we'll have a very nice user-guide for both very soon.
The future `bob.bio.base` is very different from the current one (I'll let you ...Hi guys,
We are about to have a beta release of `bob.bio.base` and `bob.bio.face`.
Thanks to @ydayer, we'll have a very nice user-guide for both very soon.
The future `bob.bio.base` is very different from the current one (I'll let you dig here https://gitlab.idiap.ch/bob/bob.bio.base/-/tree/dask-pipelines and here https://gitlab.idiap.ch/bob/bob.pipelines) and this will impact a lot the current version of this package. That's why this issue.
Until we have this package ported to use `bob.pipelines`, I would like to move the content from the current `bob.bio.base` to a module in this package, hence, we can have a beta version of the new `bob.bio.base` without breaking this one.
What do you think?Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/135New pipeline click commands are not properly implemented2021-05-19T17:12:08ZAmir MOHAMMADINew pipeline click commands are not properly implementedSee https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.extension/doc/framework.html#command-line-interfaces-with-configurations
and https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.extension/doc/py_api.html#bob...See https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.extension/doc/framework.html#command-line-interfaces-with-configurations
and https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.extension/doc/py_api.html#bob.extension.scripts.click_helper.ConfigCommand and https://www.idiap.ch/software/bob/docs/bob/docs/master/bob/bob.extension/doc/py_api.html#bob.extension.scripts.click_helper.ResourceOption
Basically the command must not run with `-p configs` it should be just `vanilla-biometrics configs`Bob 9.0.0https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/36outdated `baselines.py` script remained in setup.py2020-10-08T07:25:59ZManuel Günthersiebenkopf@googlemail.comoutdated `baselines.py` script remained in setup.pySee here: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/117ed4a764bac88c9e0d3c5b27c8c36d5cf87a61/setup.py#L106
When running `baselines.py`, I get the following error:
```
$ baselines.py -h
...
ModuleNotFoundError: No module named 'bob...See here: https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/117ed4a764bac88c9e0d3c5b27c8c36d5cf87a61/setup.py#L106
When running `baselines.py`, I get the following error:
```
$ baselines.py -h
...
ModuleNotFoundError: No module named 'bob.bio.face.script.baselines'
```Bob 9.0.0https://gitlab.idiap.ch/bob/bob.learn.mlp/-/issues/11DEPRECATION2021-09-17T14:19:31ZTiago de Freitas PereiraDEPRECATIONThis package will be deprecated in favor of Tensorflow/PyTorchThis package will be deprecated in favor of Tensorflow/PyTorchBob 9.0.0Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.ip.flandmark/-/issues/9DEPRECATION2020-11-04T11:23:18ZTiago de Freitas PereiraDEPRECATIONThis package will be deprecatedThis package will be deprecatedBob 9.0.0