beat issueshttps://gitlab.idiap.ch/groups/beat/-/issues2018-05-30T15:22:15Zhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/37`exp caches --list` doesn't print out db caches2018-05-30T15:22:15ZJaden DIEFENBAUGH`exp caches --list` doesn't print out db cachesWhen printing caches for an experiment, it prints out all blocks' caches except the dataset blocks' caches.When printing caches for an experiment, it prints out all blocks' caches except the dataset blocks' caches.https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/36Editing the json file of database views does not invalidate the cache.2020-07-28T09:45:09ZAmir MOHAMMADIEditing the json file of database views does not invalidate the cache.Steps to reproduce:
```
$ beat cache clear
$ beat exp pull amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
$ beat exp run amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
Index for database atnt/5 not found, building it
li...Steps to reproduce:
```
$ beat cache clear
$ beat exp pull amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
$ beat exp run amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
Index for database atnt/5 not found, building it
lib/python3.7/site-packages/setuptools/distutils_patch.py:26: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
"Distutils was imported before Setuptools. This usage is discouraged "
Index for database atnt/5 not found, building it
Index for database atnt/5 not found, building it
Running `amohammadi/pca/1' for block `linear_machine_training'
Start the execution of 'amohammadi/pca/1'
Block did not execute properly - outputs were reset
Standard output:
Standard error:
Captured user error:
File "lib/python3.7/site-packages/bob/io/base/__init__.py", line 143, in load
return File(inputs, 'r').read()
RuntimeError: File - constructor: C++ exception caught: 'file '/path_to_db_folder/att_faces/s1/1.pgm' is not readable'
Captured system error:
Error: Error occured: returned value is 1
Removing cache files: No data written
```
* change the json file `prefix/databases/atnt/5.json` of the database and fix the the `root_folder`
* run the experiment again:
```
$ beat exp run amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
RuntimeError: File - constructor: C++ exception caught: 'file '/path_to_db_folder/att_faces/s1/1.pgm' is not readable'
```Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/39Better copy/paste support for `exp caches`2018-06-11T08:13:15ZJaden DIEFENBAUGHBetter copy/paste support for `exp caches`Right now the caches printed are file paths:
```
block: `pre_training'
algorithm: `test/iris_preprocessor/1'
output: `/home/jae/code/beat.editor/prefix/cache/a9/e2/0b/2ba5928422351e52a508bd0cbc55e3d0e5bacb05896dc28413bcd0a1a4'
block:...Right now the caches printed are file paths:
```
block: `pre_training'
algorithm: `test/iris_preprocessor/1'
output: `/home/jae/code/beat.editor/prefix/cache/a9/e2/0b/2ba5928422351e52a508bd0cbc55e3d0e5bacb05896dc28413bcd0a1a4'
block: `pre_testing'
algorithm: `test/iris_preprocessor/1'
output: `/home/jae/code/beat.editor/prefix/cache/1e/a5/74/80c7d48bae3d0f6baf0883c3a7f78b5324830931029fcee4fd3bffff4d'
block: `training_alg'
algorithm: `test/iris_training/1'
output: `/home/jae/code/beat.editor/prefix/cache/45/61/15/3438e272598bf15e6bd729709536f4ca38b2b9d1061dd286c6dae5f288'
block: `testing_alg'
algorithm: `test/iris_testing/1'
output: `/home/jae/code/beat.editor/prefix/cache/39/d4/20/a6709aca3cac7e9a9a7cb2c36d536184554b5e19358927f2e866ad5751'
block: `analyzer'
analyzer: `test/iris_analyzer/1'
output: `/home/jae/code/beat.editor/prefix/cache/f0/a0/86/a645856dcbee4edc2729fcc60554212ed5def33334670602be6086aafb'
```
But you only use path segments relative to the cache folder to operate on that cache:
```
$ beat ca view a9/e2/0b/2ba5928422351e52a508bd0cbc55e3d0e5bacb05896dc28413bcd0a1a4
```
It would make sense for `exp caches` to only print out this bit, not the absolute path.https://gitlab.idiap.ch/beat/beat.core/-/issues/56Docker timer tests rely on having adequate CI resources2018-12-12T09:13:59ZJaden DIEFENBAUGHDocker timer tests rely on having adequate CI resourcesSometimes docker timing tests will fail when the tests go too slowly. These tests rely on a hard-coded number of seconds/milliseconds with numbers that are reasonable to assume on one's own machine but obviously aren't designed for a CI ...Sometimes docker timing tests will fail when the tests go too slowly. These tests rely on a hard-coded number of seconds/milliseconds with numbers that are reasonable to assume on one's own machine but obviously aren't designed for a CI (just see the [docker tests](https://gitlab.idiap.ch/beat/beat.core/blob/85cef289db0eac7aaa47308dd92cc5003a6a689b/beat/core/test/test_docker.py).
Some of these tests can just be changed. For example, the `test_images_cache` tests are actually relative tests - you just want to be sure that doing something a second time is faster than the first time. I believe that the CI performance would be consistent across each of these tests (which shouldn't last for more than a couple seconds).
There's other ones though, like the timeout tests. Not sure what to do with those.
This doesn't unconditionally make the CI fail since rerunning the job usually fixes it.https://gitlab.idiap.ch/beat/beat.core/-/issues/57Is `return True` needed in the algorithms API still?2018-05-30T14:33:58ZJaden DIEFENBAUGHIs `return True` needed in the algorithms API still?Not sure where I should be putting this issue...
Right now, as far as I know, `process` methods in algorithms must return `True`. Is this still required?
@samuel.gaistNot sure where I should be putting this issue...
Right now, as far as I know, `process` methods in algorithms must return `True`. Is this still required?
@samuel.gaistSamuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.web/-/issues/447add frontend regression tests2017-09-25T15:13:52ZJaden DIEFENBAUGHadd frontend regression tests99.99999999% of the frontend isn't tested right now. Adding regression tests (tests designed to be able to catch regressions/bugs in changes after the tests are added) would help with confidence in releasing new code.99.99999999% of the frontend isn't tested right now. Adding regression tests (tests designed to be able to catch regressions/bugs in changes after the tests are added) would help with confidence in releasing new code.https://gitlab.idiap.ch/beat/beat.web/-/issues/474Instructions for publishing attestations not very clear2017-09-27T14:40:18ZJaden DIEFENBAUGHInstructions for publishing attestations not very clearHello,
I got an email from beat saying that the attestation of my experiment is about to expire.
"More details: http://www.beat-eu.org/platform/attestations/232222665/"
The message informs me that if I don't take any action, the attestat...Hello,
I got an email from beat saying that the attestation of my experiment is about to expire.
"More details: http://www.beat-eu.org/platform/attestations/232222665/"
The message informs me that if I don't take any action, the attestation will be deleted [soon].
But it does not tell me what actions I could take.
The link on the "more details" line takes me to the attestation of the experiment in question.
Here again, I see the following information at the beginning of the page:
"
Created 5 months, 3 weeks ago, Jan. 5, 2017, 5:28 p.m.
Expires in 5 days, 6 hours from now, on July 4, 2017, 5:28 p.m. (unlock it to make it permanent)
Experiment: sbhatta/ivana7c/simple-antispoofing-updated/1/replay2-antispoofing-lbp-histograms
Description: Face antispoofing using comparison of LBP histograms
"
The second line says "unlock it to make it permanent", but I could not understand what "unlock it" means.
Finally, someone explained to me that I have to click on the yellow-star-wheel to "Publish" it.
Overall, I found the instructions not very clear.
I would expect the text "unlock it to make it permanent" to be an active link, which would lead to
the appropriate action.
Regards
Sushilhttps://gitlab.idiap.ch/beat/beat.web/-/issues/475New version or fork of algorithm with RST URL gets buggy documentation2017-09-27T14:52:17ZAndré AnjosNew version or fork of algorithm with RST URL gets buggy documentationIf you fork or copy an algorithm that has documentation in restructured text format containing an URL such as `My test <http://example.com>`, the copy or fork will end-up with `&lt;` and `&gt;` characters in the place of `<` and `>` and ...If you fork or copy an algorithm that has documentation in restructured text format containing an URL such as `My test <http://example.com>`, the copy or fork will end-up with `<` and `>` characters in the place of `<` and `>` and will produce an annoying warning.
Example: Try to fork `tutorial/dct/3` (https://www.beat-eu.org/platform/algorithms/tutorial/dct/3/), then, edit the fork documentation.Philip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/46plot function does not work properly if the prefix is set with relative path.2018-09-06T14:46:40ZZohreh MOSTAANIplot function does not work properly if the prefix is set with relative path.Hi,
I set my prefix to a relative path by using `beat config set` command. I can run the experiment but When I try to plot the result I get this error, it looks in the wrong directory.
```
retrieving `plotters/plotterparameters/plot/is...Hi,
I set my prefix to a relative path by using `beat config set` command. I can run the experiment but When I try to plot the result I get this error, it looks in the wrong directory.
```
retrieving `plotters/plotterparameters/plot/isoroc/1'...
Traceback (most recent call last):
File "/home/zmostaani/miniconda3/envs/beat_mini/bin/beat", line 11, in <module>
sys.exit(main())
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/decorators.py", line 75, in _decorator
value = view_func(*args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/experiments.py", line 783, in plot
output_folder
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/experiments.py", line 535, in plot_impl
data.as_dict()[the_data], output_name, None, indentation + 2, format_cache)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/plotters.py", line 238, in plot_impl
with open(output_name, "wb") as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'beat.zmostaani.prefix/prefix/beat.zmostaani.prefix/prefix/experiments/zmostaani/tutorial/eigenface/1/atnt-eigenfaces-75-comp-bis-vbox_roc.png'
```
in this example the my prefix value is `"prefix": "beat.zmostaani.prefix/prefix"` and I am running the experiment on the parent folder of beat.zmostaani.prefix.
I tried with different relative paths and I get the similar error, however if I use the absolute path everything is ok.https://gitlab.idiap.ch/beat/beat.web/-/issues/476C++ environment has no documentation2017-09-28T08:43:21ZAndré AnjosC++ environment has no documentationThe current version of the C++ environment at the production platform has no associated documentation.
We should explain there what it is and maybe have links to the different software toolkits and instructions on how to produce a binar...The current version of the C++ environment at the production platform has no associated documentation.
We should explain there what it is and maybe have links to the different software toolkits and instructions on how to produce a binary that is compatible with it.Philip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.web/-/issues/477In the experiment list view, experiments that participate on Reports are not ...2017-09-28T08:42:03ZAndré AnjosIn the experiment list view, experiments that participate on Reports are not flaggedIn the experiments list view, we have an icon that indicates if an experiment has an associated attestation-lock or a public attestation. That gives the user a notion if the assets of the said experiment are locked (frozen).
Experiments...In the experiments list view, we have an icon that indicates if an experiment has an associated attestation-lock or a public attestation. That gives the user a notion if the assets of the said experiment are locked (frozen).
Experiments that participate in a report are also "locked" (frozen assets), but that information is not shared on the experiment list view. It would be nice if one could see this information as well.Philip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.web/-/issues/480[search] Table sorting seems broken2017-12-04T10:47:47ZAndré Anjos[search] Table sorting seems brokenApparently, table sorting does not seem to work correctly at the Search web page with respect to some (numerical) columns. The table ordering seems consistent for text based columns though.
How to reproduce: If I search for "eigenface" ...Apparently, table sorting does not seem to work correctly at the Search web page with respect to some (numerical) columns. The table ordering seems consistent for text based columns though.
How to reproduce: If I search for "eigenface" then filter the search so to be able to get the EER displayed for all experiments, then click on EER does not seem to correctly rank the results.
I'm using Safari on OSX (this may be -part of- the issue).
Here is a sample of the result:
![Screen_Shot_2017-11-29_at_14.10.59](/uploads/d4c1d88295a47563aad34abb7cba53dd/Screen_Shot_2017-11-29_at_14.10.59.png)
As you can see the EER column is selected in ascending mode, but the value 0.07... comes befomre 0.06.
@philip.abbet: can you check if a fix is possible?Philip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/51The connections are not pushed to the platform when pushing a toolchain.2018-12-04T15:35:40ZZohreh MOSTAANIThe connections are not pushed to the platform when pushing a toolchain.I pushed a toolchain to the platform with this command:
```
beat toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
```
This is the output:
```
toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
[+] toolchains/tut...I pushed a toolchain to the platform with this command:
```
beat toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
```
This is the output:
```
toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
[+] toolchains/tutorial/eigenface/1 (@2014-08-11T14:27:37)
[l] toolchains/test/iris_advanced_editor/1
[l] toolchains/zmostaani/eigenface-new-error/2
[l] toolchains/tutorial/eigenface-error/1
[l] toolchains/zmostaani/editor-test/1
[l] toolchains/zmostaani/editor-test-2/1
[l] toolchains/zmostaani/eigenface-clone/1
[l] toolchains/zmostaani/eigenface-test/1
[l] toolchains/test/test/1
[l] toolchains/zmostaani/eigenface-new/1
[l] toolchains/zmostaani/sbhatta-iqm-face-antispoofing-test/1
[l] toolchains/test/iris/1
[l] toolchains/zmostaani/sbhatta-iqm-face-antispoofing-test/2
[l] toolchains/test/iris_advanced/1
[l] toolchains/zmostaani/eigenface-new-error/1
pushing toolchains/zmostaani/sbhatta-iqm-face-antispoofing-test/2 [l]
Successfuly created `zmostaani/sbhatta-iqm-face-antispoofing-test/2'
```
I went to check it on the platform. Aside from the problem with version numbering that has been mentioned in issue #49 and #50, when I look at it on the platform there is no connection to any block.https://gitlab.idiap.ch/beat/beat.web/-/issues/482[experiments] Environment "previous" relationship is not honoured by experime...2017-12-04T10:47:30ZAndré Anjos[experiments] Environment "previous" relationship is not honoured by experiment configuratorThe "Environment" objects have a "Previous Version" entry that point to environments they are suceeding. This information should be used by the experiment configurator to select reasonable defaults when the user forks an experiment that ...The "Environment" objects have a "Previous Version" entry that point to environments they are suceeding. This information should be used by the experiment configurator to select reasonable defaults when the user forks an experiment that uses outdated environments.
My recollection is that was the case, but if I fork an experiment using "Scientific Python (0.1.0)", the default selector does not point to "Scientific Python (1.0.0)". It would be great if that was the case.
Example: https://www.beat-eu.org/platform/experiments/pkorshunov/pkorshunov/speech-pad-simple/1/speech-pad_lbp_hist_ratios_lr-pa_aligned/
Just fork it and check the default environment on the Global Parameters.Philip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/53strange behaviour of beat.cmline when pulling objects from platform results i...2018-12-06T09:53:24ZZohreh MOSTAANIstrange behaviour of beat.cmline when pulling objects from platform results in beat.editor crashing as well.Hi,
Today when I ran beateditor command to edit some experiment, it did not show me anything on the Firefox. It only shows the page for split second and then it is a blank page. I updated beat.editor with conda in case there were some c...Hi,
Today when I ran beateditor command to edit some experiment, it did not show me anything on the Firefox. It only shows the page for split second and then it is a blank page. I updated beat.editor with conda in case there were some changes but the problem is still there.https://gitlab.idiap.ch/beat/beat.web/-/issues/483[experiments] Not allowed to share unfinished experiments2017-12-04T10:46:41ZPavel KORSHUNOV[experiments] Not allowed to share unfinished experimentsI forked the experiment, modified an algorithm, started the experiment, and tried to share it with @sebastien.marcel but got the error that "This experiment is not done yet", then it went to "page not found" page.
Here is the experiment...I forked the experiment, modified an algorithm, started the experiment, and tried to share it with @sebastien.marcel but got the error that "This experiment is not done yet", then it went to "page not found" page.
Here is the experiment:
https://www.beat-eu.org/platform/experiments/pkorshunov/pkorshunov/speech-pad-simple/1/speech-pad_lbp_hist_ratios_lr-pa_aligned-rr4/
Here is the error, when tried to share it:
![Screen_Shot_2017-12-04_at_11.11.13](/uploads/8e927c930758d1f1805658c26c8abf36/Screen_Shot_2017-12-04_at_11.11.13.png)https://gitlab.idiap.ch/beat/beat.web/-/issues/484[backend] Not all available resources are being utilised2017-12-04T10:47:12ZAndré Anjos[backend] Not all available resources are being utilisedFor some reason, there are experiments being scheduled and get stuck on apparent lack of resources. Meanwhile, there are idle computers on the system that seem to not take any jobs:
![image](/uploads/f0771efaf35e733b2fa76b7821a9b7f1/ima...For some reason, there are experiments being scheduled and get stuck on apparent lack of resources. Meanwhile, there are idle computers on the system that seem to not take any jobs:
![image](/uploads/f0771efaf35e733b2fa76b7821a9b7f1/image.png)Philip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/56There is no way to make a clean prefix directory structure with only trusted ...2018-12-21T14:52:06ZZohreh MOSTAANIThere is no way to make a clean prefix directory structure with only trusted system files.In the ALLIES meeting one of the points that was raised was that how to get a clean prefix directory structure, with databases, dataformats, etc in place. With only data from the system not other users, and not to go through copying a pr...In the ALLIES meeting one of the points that was raised was that how to get a clean prefix directory structure, with databases, dataformats, etc in place. With only data from the system not other users, and not to go through copying a prefix from another git repository.
It may be possible to use command line to make such a directory.https://gitlab.idiap.ch/beat/beat.core/-/issues/64Database schema and runtime improvements2021-01-29T09:46:59ZAndré AnjosDatabase schema and runtime improvementsDatabases were originally designed monolithically, assuming the environment where packages necessary to read their contents is fixed (and does not change), and the location of files to be used for the readout is fixed. As an after thoug...Databases were originally designed monolithically, assuming the environment where packages necessary to read their contents is fixed (and does not change), and the location of files to be used for the readout is fixed. As an after thought, templates were introduced without a proper formalisation and, as a consequence, templates with the same names may appear in various database JSON declarations, w/o necessarily being the same. This issue supersedes #25 and gathers all changes required for a revamp in this area:
- [x] Protocol and set templates should be separated from the database view declaration to avoid repetition and centralise "task-related" declarations. This will effectively simplify the declaration of new datasets
- [ ] The `root_db` parameter (maybe misspelled here) needs to be externalised as a runtime parameter during the run, as it is currently the case for other runtime prefixes such as algorithm caches. Currently, if one downloads the database view, they need to change this parameter by hand
- [x] The environment required to run the database view to provide the data for the experiment needs to be configured and have an entry on the database JSON declaration. This ensures that changes in the environment (docker image), will imply in new caches being generated at all times. As of today, it can happen that hashes are not regenerated even if the environment changes completely.
- [x] A default db env docker image (possibly named `beat.env.databases`) should be provided on docker hub. This avoids conflicts in future because when using multiple databases in one experiment, only one image can be used.
- [ ] The prototypes in `beat.core` should come configured to use this image.
- [ ] This new JSON entry must be documented in `beat/docs` and users should be recommended to use this image.Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/60Add support for protocol template2019-05-08T09:45:49ZSamuel GAISTAdd support for protocol templateFor database v2, the new protocol template object has been created (see beat/beat.core#64, beat/beat.core!67, and beat/beat.core!71).
This issue tracks adding support for it in beat/beat.cmdline>
Todo:
- [x] Local support
- [ ] Remote ...For database v2, the new protocol template object has been created (see beat/beat.core#64, beat/beat.core!67, and beat/beat.core!71).
This issue tracks adding support for it in beat/beat.cmdline>
Todo:
- [x] Local support
- [ ] Remote supportSamuel GAISTSamuel GAIST