beat.cmdline issueshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues2018-06-11T07:59:30Zhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/17Better cache search/discovery/browsing2018-06-11T07:59:30ZJaden DIEFENBAUGHBetter cache search/discovery/browsingRight now you have to give a path to a `.data` cache file to see anything about the cache. At the very least, I think there should be a cache listing command that gives relevant info about all the existing caches.Right now you have to give a path to a `.data` cache file to see anything about the cache. At the very least, I think there should be a cache listing command that gives relevant info about all the existing caches.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/16Add experiment results plot generation command2018-05-15T13:54:56ZJaden DIEFENBAUGHAdd experiment results plot generation commandI don't see a way to generate~visualize results from successfully-ran experiments outside of pushing the experiment to a `beat.web` installation. This feature seems practically necessary for a easy development cycle.
Maybe add a sub-com...I don't see a way to generate~visualize results from successfully-ran experiments outside of pushing the experiment to a `beat.web` installation. This feature seems practically necessary for a easy development cycle.
Maybe add a sub-command to the experiments or plotters command groups?Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/15local execution not working anymore2018-10-15T13:32:30ZJaden DIEFENBAUGHlocal execution not working anymoreNo experiments work anymore. The first block to be processed always throws the same error:
```
Running `user/integers_add/1' for block `echo'
Start the execution of 'user/integers_add/1'
Block did not execute properly - outputs were rese...No experiments work anymore. The first block to be processed always throws the same error:
```
Running `user/integers_add/1' for block `echo'
Start the execution of 'user/integers_add/1'
Block did not execute properly - outputs were reset
Standard output:
Standard error:
Captured user error:
File "/home/jae/code/beat.cmdline/src/beat.backend.python/beat/backend/python/loader.py", line 100, in run
return getattr(obj, method)(*args, **kwargs)
RuntimeError: done() takes exactly 1 argument (2 given)
Captured system error:
```
For reference - I'm using Python 2.7 via a conda environment. All packages are updated, and bob is properly installed. I used this env's python to bootstrap the buildout of `beat.cmdline`. The nosetests pass fine.Zohreh MOSTAANIZohreh MOSTAANIhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/14More detailed logging at various verbosity levels2017-08-25T07:36:23ZJaden DIEFENBAUGHMore detailed logging at various verbosity levelsRight now, the various verbosity levels don't provide much more information than not passing a verbosity level at all. Some ideas:
- More cache information
- Logging the block & its cache would be especially useful
- More statistics (...Right now, the various verbosity levels don't provide much more information than not passing a verbosity level at all. Some ideas:
- More cache information
- Logging the block & its cache would be especially useful
- More statistics (especially for the local executor)
- Printing output of every block, not just the analyserhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/13Support Python 3.x2018-05-14T12:20:39ZJaden DIEFENBAUGHSupport Python 3.xRight now buildout with python 3.x gives an error when using the `beat`.Right now buildout with python 3.x gives an error when using the `beat`.Gitlab CI Integrationhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/12Add support for multiprocessing & multi environments2017-08-24T09:20:28ZJaden DIEFENBAUGHAdd support for multiprocessing & multi environmentsRe-add/enable support for `describe` from `beat.backend.python` in `beat.cmdline` for non-docker multi-environment support.Re-add/enable support for `describe` from `beat.backend.python` in `beat.cmdline` for non-docker multi-environment support.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/11Add gridtk executor2017-08-24T08:42:54ZJaden DIEFENBAUGHAdd gridtk executorAdd another executor beside the Docker & local executors. This would also require refactoring the management of multiple executors.Add another executor beside the Docker & local executors. This would also require refactoring the management of multiple executors.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/10Add cli options for overriding parameters2017-08-24T08:41:28ZJaden DIEFENBAUGHAdd cli options for overriding parametersFor example, to override the database `atnt/3`'s root folder, you could use the cli option `--database`:
```bash
./bin/beat exp run <example> --database='atnt/3=../atnt_db'
```
This could be expanded to experiment parameters (overriding...For example, to override the database `atnt/3`'s root folder, you could use the cli option `--database`:
```bash
./bin/beat exp run <example> --database='atnt/3=../atnt_db'
```
This could be expanded to experiment parameters (overriding values defined in experiment JSON files) by giving JSON objects. The following format could be used:
```bash
./bin/beat exp run <example> --override='"globals":{"user/pca/1":{"number-of-components": 20}}'
```https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/9Un-reproducible bug -- invalid state file at `./.beat/config.json'2018-05-04T10:42:24ZTeodors EGLITISUn-reproducible bug -- invalid state file at `./.beat/config.json'I randomly get this warning:
```
WARNING: invalid state file at `./.beat/config.json' - removing and re-starting...
```
Warning is displayed (and the configuration is reset) randomly after one of these commands:
```
$ ./bin/beat config s...I randomly get this warning:
```
WARNING: invalid state file at `./.beat/config.json' - removing and re-starting...
```
Warning is displayed (and the configuration is reset) randomly after one of these commands:
```
$ ./bin/beat config set platform "http://127.0.0.1:8000/"
$ ./bin/beat config set user "user" token "3"
$ ./bin/beat config set cache "/idiap/temp/teglitis/BEAT-cache/"
$ ./bin/beat config color dark
$ ./bin/beat config list
$ ./bin/beat experiments push teodors/teodors/vein_recognition_only_dev/1/putvein-muira-only-dev-100
```
It seams that warnings aren't raised if the config is modified using single command:
```
$ ./bin/beat config set user "user" token "3" platform "http://127.0.0.1:8000/" cache "/idiap/temp/teglitis/BEAT-cache/"
```
But still this seams to me as a bug.
Teohttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/8[databases] Indexing is broken2018-04-13T08:03:56ZAndré Anjos[databases] Indexing is brokenNot all outputs are being properly indexed. To be checked ASAP.Not all outputs are being properly indexed. To be checked ASAP.Final DocumentationAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/7[experiments] Add command to checksum data and indexes2018-04-13T08:03:56ZAndré Anjos[experiments] Add command to checksum data and indexesOn ``xp caches``.On ``xp caches``.BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/6Pulling a database fails with a token error2018-04-13T08:03:56ZLaurent EL SHAFEYPulling a database fails with a token errorIt's seems that a regular user can't pull databases anymore.
```
$ ./bin/beat --prefix=prefix db pull
retrieving `databases/atnt/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: ...It's seems that a regular user can't pull databases anymore.
```
$ ./bin/beat --prefix=prefix db pull
retrieving `databases/atnt/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/banca/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/casme2/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/cbsr_nir_vis_2/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/cpqd/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/mnist/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/mobio/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/replay/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/utfvp/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/voxforge/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
retrieving `databases/xm2vts/1'...
failed to retrieve database from `https://www.beat-eu.org/platform/' with secret token, reason: Forbidden
```
This also occurs when pulling an experiment, which will try to pull the database used.BTAS Tutorialhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/5Database view is not working properly2018-04-13T08:03:56ZLaurent EL SHAFEYDatabase view is not working properlyWhen trying to view a dataset,
```
./bin/beat --prefix web_dynamic_data db view banca/1/Mc/dev_probes
```
I sometimes get this kind of errors:
```
Traceback (most recent call last):
File "./bin/beat", line 33, in <module>
...When trying to view a dataset,
```
./bin/beat --prefix web_dynamic_data db view banca/1/Mc/dev_probes
```
I sometimes get this kind of errors:
```
Traceback (most recent call last):
File "./bin/beat", line 33, in <module>
sys.exit(beat.cmdline.scripts.beat.main())
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.core/src/beat.cmdline/beat/cmdline/scripts/beat.py", line 160, in main
return module.process(args)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.core/src/beat.cmdline/beat/cmdline/databases.py", line 402, in process
return view_output(args['config'], args['<set_name>'])
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.core/src/beat.cmdline/beat/cmdline/databases.py", line 355, in view_output
json_data[name] = common.stringify(value)
TypeError: 'str' object does not support item assignment
```
BTAS TutorialLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/4Pulling an experiment does not pull any experiment but only related toolchain...2018-04-13T08:03:56ZLaurent EL SHAFEYPulling an experiment does not pull any experiment but only related toolchains/dataformats/algorithmsThis looks like API-change related.This looks like API-change related.BTAS Tutorialhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/3Data (algorithms, dataformats, etc.) pulling crashes when non-ascii character...2018-04-13T08:03:57ZLaurent EL SHAFEYData (algorithms, dataformats, etc.) pulling crashes when non-ascii characters are present in a fieldHere is an example:
```bash
lelshafey@futurix08:/idiap/user/lelshafey/work/beat_platform/beat.cmdline$ ./bin/beat algo pull
retrieving `algorithms/siebenkopf/GaborGraphSimilarity/12'...
Traceback (most recent call last):
File "....Here is an example:
```bash
lelshafey@futurix08:/idiap/user/lelshafey/work/beat_platform/beat.cmdline$ ./bin/beat algo pull
retrieving `algorithms/siebenkopf/GaborGraphSimilarity/12'...
Traceback (most recent call last):
File "./bin/beat", line 21, in <module>
sys.exit(beat.cmdline.scripts.beat.main())
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/scripts/beat.py", line 160, in main
return module.process(args)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/algorithms.py", line 144, in process
args['--force'], 0, {}, {})
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/algorithms.py", line 105, in pull
['declaration', 'code', 'description'], force, indentation)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/common.py", line 612, in pull
storage.save(**data)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/src/beat.core/beat/core/utils.py", line 262, in save
if description: self.doc.save(description)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/src/beat.core/beat/core/utils.py", line 129, in save
with open(self.path, mode) as f: f.write(contents)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 622: ordinal not in range(128)
```
This may be a problem with beat.core.BTAS TutorialLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/2Data (algorithms, dataformats, etc.) listing and pulling is not working any more2018-04-13T08:03:57ZLaurent EL SHAFEYData (algorithms, dataformats, etc.) listing and pulling is not working any moreData (algorithms, dataformats, etc.) listing and pulling is not working any more. The corresponding tests are failing as well.Data (algorithms, dataformats, etc.) listing and pulling is not working any more. The corresponding tests are failing as well.BTAS TutorialLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/1Database indexing and database view are not working anymore2018-04-13T08:03:57ZTiago de Freitas PereiraDatabase indexing and database view are not working anymoreWhen running the `beat` command line utility to perform one of the following operations (indexing or viewing database content), the script crashes with such an error:
```python
./bin/beat --prefix=work db view atnt/1/idiap/train
Tra...When running the `beat` command line utility to perform one of the following operations (indexing or viewing database content), the script crashes with such an error:
```python
./bin/beat --prefix=work db view atnt/1/idiap/train
Traceback (most recent call last):
File "./bin/beat", line 34, in <module>
sys.exit(beat.cmdline.scripts.beat.main())
File "/remote/idiap.svm/user.active/tpereira/BEAT/beat.core/src/beat.cmdline/beat/cmdline/scripts/beat.py", line 160, in main
return module.process(args)
File "/remote/idiap.svm/user.active/tpereira/BEAT/beat.core/src/beat.cmdline/beat/cmdline/databases.py", line 400, in process
return view_output(args['config'], args['<set_name>'])
File "/remote/idiap.svm/user.active/tpereira/BEAT/beat.core/src/beat.cmdline/beat/cmdline/databases.py", line 330, in view_output
input_group = view.input_group(exclude_outputs=excluded_outputs)
File "/remote/idiap.svm/user.active/tpereira/BEAT/beat.core/beat/core/database.py", line 146, in input_group
for output in self.outputs:
TypeError: 'NoneType' object is not iterable
```BTAS Tutorial