bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2022-09-22T11:53:37Zhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/86Adding PFC2022-09-22T11:53:37ZPasra RahimiAdding PFCI will try to add the PFC (With ViT backbone) to the repo, if possible, please assign me ...I will try to add the PFC (With ViT backbone) to the repo, if possible, please assign me ...Pasra RahimiPasra Rahimihttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/110Overhauling our CI and pipelines2022-11-17T17:34:29ZAndré AnjosOverhauling our CI and pipelinesI'm proposing we should re-think the whole CI procedure entirely. Our CI system was made for the time we had some C++ code to be compiled and made available for other packages (@lcolbois: this was probably another reason for the splits)...I'm proposing we should re-think the whole CI procedure entirely. Our CI system was made for the time we had some C++ code to be compiled and made available for other packages (@lcolbois: this was probably another reason for the splits).
As of now and thanks to a lot of work by predecessors, most of the packages in this group are Python-only - this is certainly true for the (biometrics) nightlies. This means we can probably benefit from this fact and improve our pipelines, at least for most packages in the bob/beat namespace.
I have a proposal in mind, but I think we need to debate this a bit more thoroughly as I may not be thinking about all aspects and oversimplifying. The bulk of the proposal goes through revising our pipeline requirements and moving most of the packages into a python-only workflow (for most part). The proposed pipeline would be like this:
1. QA via `pre-commit`, if `.pre-commit-config.yaml` is present - only linux, should fail fast. Caching is enabled.
2. Test (via pytest) on various platforms and python versions - uses a Python docker image (not conda), respects python package pinning, retrieve beta versions from internal Gitlab package registry (if they exist), otherwise PyPI. Pip caching is enabled.
3. Generate sphinx documentation building and doctests (if `doc/` directory is present) - only linux, via Python docker image (not conda)
4. Package python code (via `python setup.py sdist`)
5. Package conda package using the sdist produced at the previous step, run tests (single platform - linux, just to cross-check)
6. Deploy sdist (unreleased beta) package on internal "group" package registry (GitLab: https://docs.gitlab.com/ee/user/packages/pypi_repository/) if not tagged or private/internal. Deploy sdist package on PyPI if tagged and public.
7. Deploy conda package on internal beta channel if not tagged, deploy on stable channel if tagged.
8. Deploy documentation and coverage information on DAV web server if success (master and stable, if required)
For non-python packages, maintain minimal CI configuration, but allow maximum flexibility via their own personalised `conda_build_config.yaml`. Everything else should match the above pipeline.
I also have some ideas on how to improve the packaging:
* Moving to `pyproject.toml` (https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html), and
* Reflect that automatically on conda packages to deduplicate package lists (https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#loading-data-from-other-files)
I expect this plan should make our pipelines run much faster, while simplifying the setup and testing. Conda environment creation continues to be supported and possible. For other simpler setups, simple Python software management (e.g. via pip or poetry) would be possible.
(This would affect issue #102 for example, #104 and maybe #103.)André AnjosAndré Anjoshttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/109`bdt dev create` fails to create a Python 3.8 or 3.9 environment and installs...2022-09-15T14:52:20ZAndré Anjos`bdt dev create` fails to create a Python 3.8 or 3.9 environment and installs Python 3.10 insteadI'm not sure where the problem is, but the following doesn't seem to work anymore:
```sh
$ cd bob.extension
$ #git checkout master; git pull # just make sure you're up-to-date
$ bdt create -vv ext --python=3.8
...
$ conda activate ext
...I'm not sure where the problem is, but the following doesn't seem to work anymore:
```sh
$ cd bob.extension
$ #git checkout master; git pull # just make sure you're up-to-date
$ bdt create -vv ext --python=3.8
...
$ conda activate ext
$ python -V
Python 3.10.6
```
I have the latest beta version of bdt installed (`5.3.1b0-py39_6`, arm/mac), as it seems.
Package planning shows Python 3.8 passing by, however it is later dropped for Python 3.10.
@flavio.tarsetti, @ydayer: Does anybody understand what is going on?https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/85Formatting: output for compare_samples diagonal is not zero2022-09-12T16:20:16ZLuis LUEVANOFormatting: output for compare_samples diagonal is not zeroThe output for the compare_samples command is not zero when showing the diagonal of the All vs All comparison in all pipelines.
Bad formatting example with mobilefacenet pipeline:
```
All vs All comparison
------------------ ---------...The output for the compare_samples command is not zero when showing the diagonal of the All vs All comparison in all pipelines.
Bad formatting example with mobilefacenet pipeline:
```
All vs All comparison
------------------ -----------------------
./me.jpg ./not_me.jpg
-0.0 -0.9227539984332366
-0.922753991574597 -3.5416114485542494e-14
------------------ -----------------------
```
However it is correct with resnet50-msceleb-arcface-2021 pipeline:
```
All vs All comparison
----------------- -----------------
./me.jpg ./not_me.jpg
-0.0 -1.03846231201703
-1.03846231201703 -0.0
----------------- -----------------
```
So far I have only tested a few pipelines
- Bad formatting: facenet_sanderberg, arcface-insightface, mobilefacenet
- Correct formatting: resnet50-msceleb-arcface-2021, resnet50-msceleb-arcface20210521https://gitlab.idiap.ch/bob/bob.devtools/-/issues/108Update sphinx version to match the one available on conda-forge2022-09-02T10:43:52ZFlavio TARSETTIUpdate sphinx version to match the one available on conda-forgeThe sphinx version needs to be updated to match the version on conda-forge.
Request from @smichelThe sphinx version needs to be updated to match the version on conda-forge.
Request from @smichelRoadmap to the major version of Bob 12Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/107Add sphinx-click and sphinx-autodoc-typehints2022-09-01T14:49:11ZFlavio TARSETTIAdd sphinx-click and sphinx-autodoc-typehintsThose 2 packages will help in the development process and are requested by @smichelThose 2 packages will help in the development process and are requested by @smichelFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/187The command "bob bio compare-samples" is not working2022-08-22T09:08:59ZAlain KOMATYThe command "bob bio compare-samples" is not workingI tried following the bob.bio.base documentation and started by running the first command in the docs:
`bob bio compare-samples --pipeline facenet-sanderberg me.png not_me.png`
I got the following error:
`AttributeError: 'PipelineSimple'...I tried following the bob.bio.base documentation and started by running the first command in the docs:
`bob bio compare-samples --pipeline facenet-sanderberg me.png not_me.png`
I got the following error:
`AttributeError: 'PipelineSimple' object has no attribute 'create_biometric_reference'`Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/106Change GitLab's runners configuration to allow build and push of docker image...2022-08-19T07:44:58ZAndré MAYORAZChange GitLab's runners configuration to allow build and push of docker images using the CIIt is related to Issue [102](https://gitlab.idiap.ch/bob/bob.devtools/-/issues/102).
The goal here is to be able to build and push docker images with GitLab's ci using either directly docker or podman as follows.
```build_image:
tag...It is related to Issue [102](https://gitlab.idiap.ch/bob/bob.devtools/-/issues/102).
The goal here is to be able to build and push docker images with GitLab's ci using either directly docker or podman as follows.
```build_image:
tags:
- docker
- bob
stage: build_image
image:
name: quay.io/podman/stable
before_script:
- docker info
script:
- docker build --tag docker.idiap.ch/bob/bdt:latest .
```
It is currently impossible to do it because of access right within the container.
A change in the runner's configuration has to be made to run the container in privileged mode. An example is shown in the [GitLab documentation](https://docs.gitlab.com/runner/executors/docker.html#use-podman-to-run-docker-commands-beta).https://gitlab.idiap.ch/bob/bob.devtools/-/issues/105Pinned versions packages interpreted as float numbers2022-08-11T11:46:04ZAndré MAYORAZPinned versions packages interpreted as float numbersPinned packages listed in `conda_build_config.yaml` have their version number interpreted as floats when in format x.x and as strings when in format x.x.x when loaded by the pyyaml library in `bob.devtools/bob/devtools/build.py` in the `...Pinned packages listed in `conda_build_config.yaml` have their version number interpreted as floats when in format x.x and as strings when in format x.x.x when loaded by the pyyaml library in `bob.devtools/bob/devtools/build.py` in the `load_packages_from_conda_build_config` method.
The problem is, for instance, that if we want the package `python-graphviz=0.20` conda will search for `python-graphviz=0.2` and may return an error as it doesn't find the package in this version.
A solution could be to write all the package versions' numbers between quotes so they are all interpreted as strings.André MAYORAZAndré MAYORAZhttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/104pyproject.toml not created in new packages2022-11-07T15:58:18ZYannick DAYERpyproject.toml not created in new packagesThe required `pyproject.toml` file is not in the project template and thus not created by the `bdt dev new ...` command.The required `pyproject.toml` file is not in the project template and thus not created by the `bdt dev new ...` command.https://gitlab.idiap.ch/bob/bob.paper.8years/-/issues/5Provide shore files and instructions on how to use them2022-08-11T12:57:54ZManuel Günthersiebenkopf@googlemail.comProvide shore files and instructions on how to use themFor easy reproduction of our plots, score files would be a great asset. Maybe it is possible to upload them to the Biometrics Resources (https://www.idiap.ch/webarchives/sites/www.idiap.ch/resource/biometric/) where the score files for t...For easy reproduction of our plots, score files would be a great asset. Maybe it is possible to upload them to the Biometrics Resources (https://www.idiap.ch/webarchives/sites/www.idiap.ch/resource/biometric/) where the score files for the old paper also are.Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.paper.8years/-/issues/4Instructions for GBU plots are missing in the README2022-08-10T09:42:39ZManuel Günthersiebenkopf@googlemail.comInstructions for GBU plots are missing in the READMEThe README lists 6 datasets to be used, but provides commands only for 5 of them. As far as I can see, the GBU dataset is missing.The README lists 6 datasets to be used, but provides commands only for 5 of them. As far as I can see, the GBU dataset is missing.Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.devtools/-/issues/103CI fails on macos-arm64 only packages2022-09-20T09:35:21ZYannick DAYERCI fails on macos-arm64 only packagesDefining the pins for the `mne` package (added by !307) fails on [`build_macos_arm_bob_devtools`](https://gitlab.idiap.ch/bob/bob.devtools/-/jobs/277736).
`mne` has different dependencies depending on the architecture, and needs `pyobjc...Defining the pins for the `mne` package (added by !307) fails on [`build_macos_arm_bob_devtools`](https://gitlab.idiap.ch/bob/bob.devtools/-/jobs/277736).
`mne` has different dependencies depending on the architecture, and needs `pyobjc-framework-cocoa` on mac machines.
However, it is not found when running on a mac arm CI pipeline: `package mne-1.1.0-hce30654_0 requires pyobjc-framework-cocoa, but none of the providers can be installed`
- Trying to install `mne` directly on the mac arm machine works and mamba finds `pyobjc-framework-cocoa`.
- [The other CI pipelines](https://gitlab.idiap.ch/bob/bob.devtools/-/pipelines/63615) pass.
**Assumption**: Our CI configuration for macos arm is not correct and somehow does not search for packages on the `osx-arm64` platform on conda-forge.https://gitlab.idiap.ch/bob/bob.devtools/-/issues/102Improve pipelines by using custom Docker images2022-10-05T22:46:25ZSamuel GAISTImprove pipelines by using custom Docker imagesThe current pipeline of BOB, and therefore BEAT, follows this schema:
- Grab the bootstrap script
- Bootstrap miniconda (which might be downloading the installer script)
- Setup some configuration
- Do some checks
- Do a cleanup
So ev...The current pipeline of BOB, and therefore BEAT, follows this schema:
- Grab the bootstrap script
- Bootstrap miniconda (which might be downloading the installer script)
- Setup some configuration
- Do some checks
- Do a cleanup
So every build starts from scratch, installs Miniconda, which version is hard coded in `bootstrap.py`, then downloads a bunch of dependencies, do the actual work, rinse and repeat for each and every package. Even if there is some caching involved it is still pretty inefficient in terms of bandwidth, power and time consumption.
The script is more or less optimized for shell runner usage however, all these runners can run Docker and actually do use the Docker runner for the Linux build stage.
Since BOB pins its dependencies pretty strictly, it would make sense to create a Docker image with the environment preinstalled. That would reduce that repetitive bdt environment setup happening for every job.
What I would suggest is to do that in steps:
1) Create a bdt Docker image based on the image used for the Linux build jobs
2) Use that image for Linux specific and deploy job
3) Cache the conda related downloads
4) Check if it would make sense to preinstall in the Docker image all the bob pinned dependencies
5) Check if the [Docker-OSX project](https://hub.docker.com/r/sickcodes/docker-osx) is something that could be used on the macOS runners so that we don't need a shell runner anymore
6) If above's answer is yes, create a macOS bdt image keeping in mind point 4
7) Use Docker for all stages
For point 4, even if we have a local mirror for conda, does it really make sense to re-download everything every time ?Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/84[RFC] PyTorchModel2023-01-25T08:30:14ZChristophe ECABERT[RFC] PyTorchModelWith current implementation of the `PyTorchModel` the weights and the architecture need to be provided through `checkpoint_path` and `config` in order to use the transformer. This constraint can be alleviate using the `TorchScript` scrip...With current implementation of the `PyTorchModel` the weights and the architecture need to be provided through `checkpoint_path` and `config` in order to use the transformer. This constraint can be alleviate using the `TorchScript` script feature [ref](https://pytorch.org/docs/stable/jit.html).
TorchScript convert any `torch.nn.Module` into a persistent executable that can be loaded and used directly without needing to build the architecture first. It basically saves the "code" and the weights into a single file in the same fashion as Tensorflow. Moreover, some optimizations can be turned on during the saving phase such as converting all the ops into constant ops, freezing graph and so on.
Such mechanism can be used in the `PyTorchModel` base class to greatly simplify how we add new models. An example is provided below:
```python
class TorchScriptModel(TransformerMixin, BaseEstimator):
def __init__(self,
model_path,
preprocessor,
memory_demanding=False,
device=None,
**kwargs):
super().__init__(**kwargs)
self.memory_demanding = memory_demanding
# Model
self.model_path = model_path
self.model = None
self.preprocessor = preprocessor
if device is None:
device = pt.device('cpu')
self.device = device
def _model(self):
if self.model is None:
# For now, we suggest to disable the Jit Autocast Pass,
# As the issue: https://github.com/pytorch/pytorch/issues/75956
pt._C._jit_set_autocast_mode(False)
self.model = pt.jit.load(self.model_path)
self.model.eval()
self.model.to(self.device)
return self.model
def transform(self, X):
X = check_array(X, allow_nd=True).astype(np.float32)
model = self._model()
def _transform(x):
x = pt.from_numpy(x)
with pt.no_grad():
# Preprocess
x = x.to(self.device)
x = self.preprocessor(x)
# Extract embedding
x = model(x)
return x.cpu().numpy()
if self.memory_demanding:
features = []
for x in X:
f = _transform(x[None, ...])
features.append(f)
features = np.asarray(features)
if features.ndim >= 3:
features = np.vstack(features)
return features
else:
return _transform(X)
def __getstate__(self):
# Handling unpicklable objects
d = {}
for key, value in self.__dict__.items():
if key != 'model':
d[key] = value
d['model'] = None
return d
def _more_tags(self):
return {"requires_fit": False}
```
Moreover, some architecture are defined as a derived class of `PyTorchModel` such as `IResnetXXX` whereas others are defined through function returning a `PipelineSimple` instance. The consistency could be improved for instance by using `classmethod` to create a certain type of architecture. For instance it could be something like:
```python
class TorchScriptModel(TransformerMixin, BaseEstimator):
@classmethod
def IResNet(cls, version: Enum):
# Mechanic to retrive model from idiap server
return cls(...)
```
What are your thoughts on this @ydayer, @flavio.tarsetti, @lcolbois ?https://gitlab.idiap.ch/bob/bob.bio.spear/-/issues/43SpeechbrainEmbeddings fails when run in dask for the first time2022-07-21T09:09:34ZYannick DAYERSpeechbrainEmbeddings fails when run in dask for the first timeWhen running `speechbrain-ecapa-voxceleb` for the first time, it fails if on dask.
Multiple workers are calling load_model at the same time and as the model files do not exist (first time running the pipeline), speechbrain tries to down...When running `speechbrain-ecapa-voxceleb` for the first time, it fails if on dask.
Multiple workers are calling load_model at the same time and as the model files do not exist (first time running the pipeline), speechbrain tries to download them from huggingface. It uses a cache in `~/.cache/huggingface`. Error rise as multiple worker access simultaneousely to the same files.
Workaround: The first time you run `speechbrain-ecapa-voxceleb`, do it without the `-l ...` option, at least until the files are downloaded and the computations starts. You can then stop the execution and run with the `-l` option.Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/27`all_samples` in `full` protocol of utfvp dataset2022-08-10T08:28:12ZHatef OTROSHI`all_samples` in `full` protocol of utfvp datasetHi,
It seems that `all_samples` in `full` protocol of the utfvp dataset has a problem and raises an issue.
Here is a sample code:
```
from bob.bio.vein.database.utfvp import UtfvpDatabase
database = UtfvpDatabase(protocol="full")
datab...Hi,
It seems that `all_samples` in `full` protocol of the utfvp dataset has a problem and raises an issue.
Here is a sample code:
```
from bob.bio.vein.database.utfvp import UtfvpDatabase
database = UtfvpDatabase(protocol="full")
database.all_samples()
```
and here is the error:
```
packages/bob/bio/base/database/csv_dataset.py", line 494, in all_samples
samples = samples + self.background_model_samples()
File "..../lib/python3.9/site-packages/bob/bio/base/database/csv_dataset.py", line 381, in background_model_samples
self.csv_to_sample_loader.transform(self.train_csv)
File "..../lib/python3.9/site-packages/sklearn/pipeline.py", line 635, in transform
Xt = transform.transform(Xt)
File "..../lib/python3.9/site-packages/bob/pipelines/sample_loaders.py", line 84, in transform
X.seek(0)
AttributeError: 'NoneType' object has no attribute 'seek'
```https://gitlab.idiap.ch/bob/bob.bio.face/-/issues/83Leaderboad needs to point to bob.bio.face_ongoing's temporarily2022-08-10T10:27:03ZYannick DAYERLeaderboad needs to point to bob.bio.face_ongoing's temporarilyWhile the leaderboard is being re-built, we should point to the old existing one in bob.bio.face_ongoing [here](https://www.idiap.ch/software/bob/docs/bob/bob.bio.face_ongoing/master/leaderboard.html).
or include it in this doc, since t...While the leaderboard is being re-built, we should point to the old existing one in bob.bio.face_ongoing [here](https://www.idiap.ch/software/bob/docs/bob/bob.bio.face_ongoing/master/leaderboard.html).
or include it in this doc, since the command lines shown there are no longer working.Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.face/-/issues/82RFW dataset: overlapping and mis-labelling between training and testing sets2022-07-13T13:01:02ZYu LinghuRFW dataset: overlapping and mis-labelling between training and testing setsBased on the datasets we received from Wang et al., when we use z-samples or t-samples as shown below
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/master/bob/bio/face/database/rfw.py#L242, 2 problems occurred during the experiments.
...Based on the datasets we received from Wang et al., when we use z-samples or t-samples as shown below
https://gitlab.idiap.ch/bob/bob.bio.face/-/blob/master/bob/bio/face/database/rfw.py#L242, 2 problems occurred during the experiments.
1. There are 44 subjects classified as Caucasian in the training set, but as Indian in the testing set. (e.g. m.0c96fs, m.08y5xt, etc.)
2. When we choose to obtain 2500 z-samples from each race as the cohort, we detect more than 6000 pairs of subjects (one from training and one from testing) that have very high similarity scores (-0.5~-0.1). After manually check some of them, those samples should belong to same person, i.e. not imposter scores. So the overlapping exists between training and testing sets, which is not supposed to be.
This bug report works as a record of problems. I'm not sure if those problems only happen to us because of different versions of datasets.
We could discuss it in a later stage, e.g. use other BUPT datasets like BUPT-Balanced as training set since Wang et al. stated there is no overlap between BUPT-Balanced and RFW, face detection might be necessary since no landmark is given for this dataset.https://gitlab.idiap.ch/bob/bob.bio.base/-/issues/186Migrating the database interfaces to the new new CSV format2022-09-06T13:19:13ZAmir MOHAMMADIMigrating the database interfaces to the new new CSV formatA new interface is being implemented in https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/300, and I am trying to migrate all dbs to that.
This is a meta issue to track the migration.
- [ ] Convert all csv based ones to the new ...A new interface is being implemented in https://gitlab.idiap.ch/bob/bob.bio.base/-/merge_requests/300, and I am trying to migrate all dbs to that.
This is a meta issue to track the migration.
- [ ] Convert all csv based ones to the new format
- [ ] Convert all custom interfaces to the new format
- [ ] Rewrite `bob.bio.face.database` file for database that were using custom interface
- [ ] Test and verify the new interfaces!
- [ ] Upload csv files according to https://gitlab.idiap.ch/bob/private/-/wikis/How-to-upload-resources
- follow the `data/bob/bob.bio.modality/bio-modality-database_name.tar.gz`
- Use this format in the code:
```python
from bob.extension.download import get_file
name = "bio-modality-database_name-586b7e81.tar.gz"
dataset_protocols_path = get_file(
name,
# don't use https here, use http so the link works in the CI as well.
[f"https://www.idiap.ch/software/bob/data/bob/bob.bio.modality/{name}",
f"http://www.idiap.ch/software/bob/data/bob/bob.bio.modality/{name}"],
cache_subdir="protocols",
file_hash="586b7e81",
)
# remove .urls method.
```
Separate problems
- [ ] asvspoof2017-spoof the interface does not load
- [ ] utfvp has some columns without header in its CSV files!
- [ ] caspeal has some columns without header in its CSV files!
- [ ] gbu, rfw, lfw, ijbc, youtube interfaces are custom
- [ ] bob.bio.spear licit and spoof protocols need to change to new format like replaymobileLaurent COLBOISLaurent COLBOIS