bob issueshttps://gitlab.idiap.ch/groups/bob/-/issues2022-03-24T18:21:54Zhttps://gitlab.idiap.ch/bob/bob.learn.em/-/issues/38More dask tests2022-03-24T18:21:54ZAmir MOHAMMADIMore dask tests- [ ] There are no dask array as input tests for gmms
- [ ] Dask tests should run under multiprocessing of distributed package to make sure real-world conditions are simulated.- [ ] There are no dask array as input tests for gmms
- [ ] Dask tests should run under multiprocessing of distributed package to make sure real-world conditions are simulated.Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.learn.em/-/issues/37Excess memory usage in kmeans training2022-03-22T17:34:30ZAmir MOHAMMADIExcess memory usage in kmeans trainingWhen training on voxforge and 256 GMMs with dask partition size of 200, I get this error:
```
bob.learn.em/bob/learn/em/k_means.py", line 78, in e_step
np.eye(n_clusters)[closest_k_indices][:, :, None] * data[:, None],
numpy.core._ex...When training on voxforge and 256 GMMs with dask partition size of 200, I get this error:
```
bob.learn.em/bob/learn/em/k_means.py", line 78, in e_step
np.eye(n_clusters)[closest_k_indices][:, :, None] * data[:, None],
numpy.core._exceptions.MemoryError: Unable to allocate 7.39 GiB for an array with shape (64546, 256, 60) and data type float64
```Yannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.learn.em/-/issues/36get_centroids_distance gets called twice during e_step in kmeans2022-03-21T10:48:27ZAmir MOHAMMADIget_centroids_distance gets called twice during e_step in kmeanshttps://gitlab.idiap.ch/bob/bob.db.base/-/issues/28Deprecate this package2022-04-27T20:13:01ZAmir MOHAMMADIDeprecate this packageThere is only `bob.db.atnt` left, after that, we can deprecate this package.There is only `bob.db.atnt` left, after that, we can deprecate this package.The Great Deprecationhttps://gitlab.idiap.ch/bob/bob.pad.base/-/issues/40Refactor vanilla pad script as reusable functions2022-03-03T17:24:16ZAmir MOHAMMADIRefactor vanilla pad script as reusable functionsAmir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.extension/-/issues/83Matplotlib base documentation directory has changed2022-03-01T08:23:13ZAndré AnjosMatplotlib base documentation directory has changedWe need to modify the base URL for the matplotlib manual at https://gitlab.idiap.ch/bob/bob.extension/-/blob/master/bob/extension/utils.py#L527 from `https://matplotlib.org/` to `https://matplotlib.org/stable/`. Builds are breaking on t...We need to modify the base URL for the matplotlib manual at https://gitlab.idiap.ch/bob/bob.extension/-/blob/master/bob/extension/utils.py#L527 from `https://matplotlib.org/` to `https://matplotlib.org/stable/`. Builds are breaking on this account:
https://gitlab.idiap.ch/bob/bob.ip.binseg/-/jobs/258731https://gitlab.idiap.ch/bob/bob.ip.facedetect/-/issues/13Nightlies failing because of this one2022-02-28T14:34:46ZTiago de Freitas PereiraNightlies failing because of this oneNow it is something related with TF
https://gitlab.idiap.ch/bob/nightlies/-/jobs/258558Now it is something related with TF
https://gitlab.idiap.ch/bob/nightlies/-/jobs/258558https://gitlab.idiap.ch/bob/bob.devtools/-/issues/93Missing CUDA overrides2022-02-28T14:34:46ZAmir MOHAMMADIMissing CUDA overridesJob [#258340](https://gitlab.idiap.ch/bob/bob.devtools/-/jobs/258340) failed for 93acc950ec94de1ff7d4629d8bdfbe9cf8c25c77:Job [#258340](https://gitlab.idiap.ch/bob/bob.devtools/-/jobs/258340) failed for 93acc950ec94de1ff7d4629d8bdfbe9cf8c25c77:Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.learn.em/-/issues/35Small changes to the new API needed2022-02-24T17:21:38ZAmir MOHAMMADISmall changes to the new API needed- [ ] rename kmeans_trainer parameter in GMMMachine to `kmeans_machine`
- [ ] make the Kmeans and GMM machines available in `bob.learn.em`.
- ~~[ ] The `__call__` methods are missing.~~
- [ ] load methods are missing
- [ ] setup.py requi...- [ ] rename kmeans_trainer parameter in GMMMachine to `kmeans_machine`
- [ ] make the Kmeans and GMM machines available in `bob.learn.em`.
- ~~[ ] The `__call__` methods are missing.~~
- [ ] load methods are missing
- [ ] setup.py requirements.txt and meta.yaml is not updated.
Some other things that I will add to this issue.
The changes don't need to be addressed now.https://gitlab.idiap.ch/bob/bob.fusion.base/-/issues/9Needs to be ported with the new bob.learn.em API2022-03-02T09:27:39ZAmir MOHAMMADINeeds to be ported with the new bob.learn.em APIJob [#258101](https://gitlab.idiap.ch/bob/bob.fusion.base/-/jobs/258101) failed for 506b8b66a584ffb37d6038b6a87a5e40510f2142:Job [#258101](https://gitlab.idiap.ch/bob/bob.fusion.base/-/jobs/258101) failed for 506b8b66a584ffb37d6038b6a87a5e40510f2142:Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.vein/-/issues/26Fails on MacOS Arm with AttributeError: module 'numpy' has no attribute 'floa...2022-03-03T14:22:23ZAmir MOHAMMADIFails on MacOS Arm with AttributeError: module 'numpy' has no attribute 'float128'Job [#256859](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/256859) failed for 34ddbc6fb35136fcd3cdf7ad6b87296c023b2c64:
```
======================================================================
ERROR: bob.bio.vein.tests.test.test_mas...Job [#256859](https://gitlab.idiap.ch/bob/bob.bio.vein/-/jobs/256859) failed for 34ddbc6fb35136fcd3cdf7ad6b87296c023b2c64:
```
======================================================================
ERROR: bob.bio.vein.tests.test.test_mask_to_image
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/gitlab/builds/sAZPGTqW/0/bob/bob.bio.vein/miniconda/conda-bld/bob.bio.vein_1644593419589//lib/python3.9/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/Users/gitlab/builds/sAZPGTqW/0/bob/bob.bio.vein/miniconda/conda-bld/bob.bio.vein_1644593419589//lib/python3.9/site-packages/bob/bio/vein/tests/test.py", line 474, in test_mask_to_image
_check_float(32)
File "/Users/gitlab/builds/sAZPGTqW/0/bob/bob.bio.vein/miniconda/conda-bld/bob.bio.vein_1644593419589//lib/python3.9/site-packages/bob/bio/vein/tests/test.py", line 470, in _check_float
conv = preprocessor_utils.mask_to_image(sample, 'float%d' % n)
File "/Users/gitlab/builds/sAZPGTqW/0/bob/bob.bio.vein/miniconda/conda-bld/bob.bio.vein_1644593419589//lib/python3.9/site-packages/bob/bio/vein/preprocessor/utils.py", line 158, in mask_to_image
elif dtype in (numpy.float32, numpy.float64, numpy.float128):
File "/Users/gitlab/builds/sAZPGTqW/0/bob/bob.bio.vein/miniconda/conda-bld/bob.bio.vein_1644593419589//lib/python3.9/site-packages/numpy/__init__.py", line 313, in __getattr__
raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'float128'
```https://gitlab.idiap.ch/bob/bob.learn.em/-/issues/34Does not work with h5py 32022-02-21T18:23:19ZAmir MOHAMMADIDoes not work with h5py 3Job [#257793](https://gitlab.idiap.ch/bob/bob.learn.em/-/jobs/257793) failed for 462d8bda27b8bef01a11f187fbb7ade153a650a0:Job [#257793](https://gitlab.idiap.ch/bob/bob.learn.em/-/jobs/257793) failed for 462d8bda27b8bef01a11f187fbb7ade153a650a0:Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.learn.em/-/issues/33Nightlies failing because of this one2022-02-22T13:48:19ZTiago de Freitas PereiraNightlies failing because of this oneNightlies is failing because of this one.
The issue seems related to HDF5
https://gitlab.idiap.ch/bob/nightlies/-/jobs/257644
https://gitlab.idiap.ch/bob/bob.learn.em/-/jobs/257680
Could you please look at that @ydayer ?
ThanksNightlies is failing because of this one.
The issue seems related to HDF5
https://gitlab.idiap.ch/bob/nightlies/-/jobs/257644
https://gitlab.idiap.ch/bob/bob.learn.em/-/jobs/257680
Could you please look at that @ydayer ?
ThanksYannick DAYERYannick DAYERhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/175Script for feature extraction from database2022-03-19T11:10:26ZManuel Günthersiebenkopf@googlemail.comScript for feature extraction from databaseIn many cases, we would just want to have a script to extract the features for all samples of our database (using a specifiable ``Transformer``), so that we can use them in a different process. Currently, there is no such script availabl...In many cases, we would just want to have a script to extract the features for all samples of our database (using a specifiable ``Transformer``), so that we can use them in a different process. Currently, there is no such script available.
I would propose to add a script as follows:
```
import argparse
import os
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
description='Extract features from the given dataset'
)
parser.add_argument("--transformer", "-e", required=True, help="Select the transformer to be used")
parser.add_argument("--dataset", "-d", required=True, help="Select the dataset from which to extract features")
parser.add_argument("--output-directory", "-o", required=True, help="Select the directory where to write the data to")
args = parser.parse_args()
import bob.bio.base
import bob.core
import bob.io.base
logger = bob.core.log.setup("bob.paper.osijbc")
bob.core.log.set_verbosity_level(logger, 2)
database = bob.bio.base.load_resource(args.dataset, "database")
transformer = bob.bio.base.load_resource(args.transformer, "transformer")
for idx, samples in enumerate(database.all_samples()):
logger.info('Extracting features for sample', )
features = transformer.transform(samples)
for feature in features:
output = os.path.join(args.output_directory, feature.key + ".hdf5")
logger.debug('Writing file', output)
bob.io.base.save(feature.data, output, True)
```
To be consistent with our other scripts, I would recommend to use `click` instead of `argparse`. Unfortunately, I am not familiar with click and I have no time to learn how to implement click commands right now. Would anyone else -- with more experience with `click` take this over?Tiago de Freitas PereiraTiago de Freitas Pereirahttps://gitlab.idiap.ch/bob/bob.io.base/-/issues/22Does not compile with hdf5 1.122022-02-21T11:37:29ZTiago de Freitas PereiraDoes not compile with hdf5 1.12Compilation issue with HDF5
https://gitlab.idiap.ch/bob/nightlies/-/jobs/257523/raw
Probably related to these bumps
https://gitlab.idiap.ch/bob/bob.devtools/-/merge_requests/273
```
/scratch/builds/bob/nightlies/src/bob/bob.io.base/...Compilation issue with HDF5
https://gitlab.idiap.ch/bob/nightlies/-/jobs/257523/raw
Probably related to these bumps
https://gitlab.idiap.ch/bob/bob.devtools/-/merge_requests/273
```
/scratch/builds/bob/nightlies/src/bob/bob.io.base/bob/io/base/cpp/HDF5Group.cpp: In member function 'herr_t bob::io::base::detail::hdf5::Group::iterate_callback(hid_t, const char*, const H5L_info2_t*)':
/scratch/builds/bob/nightlies/src/bob/bob.io.base/bob/io/base/cpp/HDF5Group.cpp:88:73: error: too few arguments to function 'herr_t H5Oget_info_by_name3(hid_t, const char*, H5O_info2_t*, unsigned int, hid_t)'
88 | herr_t status = H5Oget_info_by_name(self, name, &obj_info, H5P_DEFAULT);
| ^
In file included from /scratch/builds/bob/nightlies/miniconda/conda-bld/bob.io.base_1645283626826/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/include/H5Apublic.h:22,
from /scratch/builds/bob/nightlies/miniconda/conda-bld/bob.io.base_1645283626826/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/include/hdf5.h:23,
from /scratch/builds/bob/nightlies/src/bob/bob.io.base/bob/io/base/include/bob.io.base/HDF5Group.h:15,
from /scratch/builds/bob/nightlies/src/bob/bob.io.base/bob/io/base/cpp/HDF5Group.cpp:18:
/scratch/builds/bob/nightlies/miniconda/conda-bld/bob.io.base_1645283626826/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/include/H5Opublic.h:557:15: note: declared here
557 | H5_DLL herr_t H5Oget_info_by_name3(hid_t loc_id, const char *name, H5O_info2_t *oinfo, unsigned fields,
| ^~~~~~~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/bob_io_base.dir/build.make:160: CMakeFiles/bob_io_base.dir/scratch/builds/bob/nightlies/src/bob/bob.io.base/bob/io/base/cpp/HDF5Group.cpp.o] Error 1
```Amir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.pad.face/-/issues/43Nightlies failing2022-02-22T17:55:07ZTiago de Freitas PereiraNightlies failingNightlies failing because of this one
https://gitlab.idiap.ch/bob/nightlies/-/jobs/257429
Can someone have a look on that?
Thanks
ping @amohammadi @ageorgeNightlies failing because of this one
https://gitlab.idiap.ch/bob/nightlies/-/jobs/257429
Can someone have a look on that?
Thanks
ping @amohammadi @ageorgeAmir MOHAMMADIAmir MOHAMMADIhttps://gitlab.idiap.ch/bob/bob.bio.base/-/issues/174Nightlies failing here2022-02-16T09:30:59ZTiago de Freitas PereiraNightlies failing herePLDA was temporarily removed from `bob.learn.em` with https://gitlab.idiap.ch/bob/bob.learn.em/-/merge_requests/42
we need to remove it from here too.PLDA was temporarily removed from `bob.learn.em` with https://gitlab.idiap.ch/bob/bob.learn.em/-/merge_requests/42
we need to remove it from here too.https://gitlab.idiap.ch/bob/bob.ip.facedetect/-/issues/12Nightlies failling here2022-02-15T15:40:38ZTiago de Freitas PereiraNightlies failling hereSomething with boosting.
I don't understand why this is failing now.
Needs investigation
https://gitlab.idiap.ch/bob/nightlies/-/pipelines/58308Something with boosting.
I don't understand why this is failing now.
Needs investigation
https://gitlab.idiap.ch/bob/nightlies/-/pipelines/58308https://gitlab.idiap.ch/bob/bob.paper.8years/-/issues/3Move plots to bob.bio.face2022-08-08T10:59:50ZTiago de Freitas PereiraMove plots to bob.bio.faceHi @mguenther,
We've made in this package several customized plots.
Can I move those customized plots to `bob.bio.face`?
ThanksHi @mguenther,
We've made in this package several customized plots.
Can I move those customized plots to `bob.bio.face`?
Thankshttps://gitlab.idiap.ch/bob/bob.io.video/-/issues/20Nightlies failing ON ARM because of this one.2022-01-19T13:08:57ZTiago de Freitas PereiraNightlies failing ON ARM because of this one.I think we need make this guy with "!" like the other ones until we decide to fix issues on this platform.I think we need make this guy with "!" like the other ones until we decide to fix issues on this platform.