beat.cmdline issueshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues2020-05-26T15:19:19Zhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/38Offline plotting support is shaky2020-05-26T15:19:19ZJaden DIEFENBAUGHOffline plotting support is shakyIf the plotter object and the default plotterparameters exist in the local prefix, failing to fetch info from the BEAT web platform should not stop plotting.
The issue came up when using the tutorial VM. Some people didn't have an inter...If the plotter object and the default plotterparameters exist in the local prefix, failing to fetch info from the BEAT web platform should not stop plotting.
The issue came up when using the tutorial VM. Some people didn't have an internet connection and couldn't plot because `beat.cmdline` was trying to fetch stuff from beat.web even though the plotter objects and default parameters existed.Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/37`exp caches --list` doesn't print out db caches2018-05-30T15:22:15ZJaden DIEFENBAUGH`exp caches --list` doesn't print out db cachesWhen printing caches for an experiment, it prints out all blocks' caches except the dataset blocks' caches.When printing caches for an experiment, it prints out all blocks' caches except the dataset blocks' caches.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/36Add `--long` flag when listing things2018-05-30T08:28:19ZJaden DIEFENBAUGHAdd `--long` flag when listing thingsIt's just like `ls`'s `-l` flag:
- last edited
- maybe which filetypes the object has associated with it?It's just like `ls`'s `-l` flag:
- last edited
- maybe which filetypes the object has associated with it?https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/35Add sort option for `beat <subcommand> list`2018-05-30T08:24:38ZJaden DIEFENBAUGHAdd sort option for `beat <subcommand> list`Sort by:
- Name
- Last edited
- ???Sort by:
- Name
- Last edited
- ???https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/34`beat caches view` doesn't work2018-12-12T09:14:17ZJaden DIEFENBAUGH`beat caches view` doesn't workI get the following:
```
path: e4/fb/dc/b2f4751a65179e51b472c4b8c1efbec86441e61194bc02442971ae3405
dataformat: tutorial/linear_machine/1
Traceback (most recent call last):
File "/home/jae/miniconda3/envs/beatedit/bin/beat", line 11, ...I get the following:
```
path: e4/fb/dc/b2f4751a65179e51b472c4b8c1efbec86441e61194bc02442971ae3405
dataformat: tutorial/linear_machine/1
Traceback (most recent call last):
File "/home/jae/miniconda3/envs/beatedit/bin/beat", line 11, in <module>
sys.exit(main())
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/cmdline/scripts/beat.py", line 186, in main
return module.process(args)
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/cmdline/cache.py", line 191, in process
return view(args['config'], args['<path>'], start, end)
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/cmdline/cache.py", line 164, in view
cls=NumpyJSONEncoder).\
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/__init__.py", line 397, in dumps
**kw).encode(obj)
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 293, in encode
chunks = list(chunks)
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 677, in _iterencode
for chunk in _iterencode_dict(o, _current_indent_level):
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 634, in _iterencode_dict
for chunk in chunks:
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 697, in _iterencode
o = _default(o)
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/backend/python/utils.py", line 354, in default
return simplejson.JSONEncoder.default(self, obj)
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 268, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <map object at 0x7f1b3d54a4e0> is not JSON serializable
```
This happens with any cache I've tried. Does it work for anyone?Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/33`edit` command fails with "File does not exist"2018-05-28T14:56:24ZJaden DIEFENBAUGH`edit` command fails with "File does not exist"I wanted to update my slides to use the new `edit` subcommand, but trying to use `beat <entity> edit <name>` fails for everything I've tried.
I tried
```
beat exp edit test/test/iris/1/iris
```
and I get:
```
File does not exist: /hom...I wanted to update my slides to use the new `edit` subcommand, but trying to use `beat <entity> edit <name>` fails for everything I've tried.
I tried
```
beat exp edit test/test/iris/1/iris
```
and I get:
```
File does not exist: /home/jae/code/beat.editor/prefix/experiments/test/test/iris/1/iris.json
```
However, `cat`ing that absolute path works:
```
cat /home/jae/code/beat.editor/prefix/experiments/test/test/iris/1/iris.json
```
prints the file contents.
Same thing with `beat alg edit test/iris_analyzer/1`, giving:
```
File does not exist: /home/jae/code/beat.editor/prefix/algorithms/test/iris_analyzer/1.py
```
and `beat lib edit plot/baselib/1` giving:
```
File does not exist: /home/jae/code/beat.editor/prefix/libraries/plot/baselib/1.py
```
All of these files do actually exist and `cat`ing them using that given file path works.
@flavio.tarsetti @samuel.gaisthttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/32Plots from different experiments using the same username, toolchain, and anal...2018-05-25T08:12:13ZJaden DIEFENBAUGHPlots from different experiments using the same username, toolchain, and analyzer output names overwrite eachotherSay there are two experiments, `test/test/iris/1/iris` and `test/test/iris/1/means`. The experiment metadata for each of these experiments will be found in the same folder, `test/test/iris/1`. Say that these experiments use the same anal...Say there are two experiments, `test/test/iris/1/iris` and `test/test/iris/1/means`. The experiment metadata for each of these experiments will be found in the same folder, `test/test/iris/1`. Say that these experiments use the same analyzer as well.
If you plot these experiments , the last one to be plotted will overwrite the other experiment's plot files. This is because there is no per-experiment unique key added to the plot image file names - instead the file names are only based off of the output names in the analyzer.Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/31Edit command for all packages2018-10-15T07:53:54ZFlavio TARSETTIEdit command for all packagesAdding the edit command for all packages.
This is related to the discussion here https://gitlab.idiap.ch/beat/beat.cmdline/merge_requests/29Adding the edit command for all packages.
This is related to the discussion here https://gitlab.idiap.ch/beat/beat.cmdline/merge_requests/29Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/30[ci] issue with docker2018-05-10T12:13:54ZFlavio TARSETTI[ci] issue with dockerHi @andre.anjos ,
Do you see why docker is causing issues here ?
https://gitlab.idiap.ch/beat/beat.cmdline/pipelines/19968
ThanksHi @andre.anjos ,
Do you see why docker is causing issues here ?
https://gitlab.idiap.ch/beat/beat.cmdline/pipelines/19968
ThanksAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/29Provide a plugin-based command line mechanism2020-07-21T14:02:10ZAmir MOHAMMADIProvide a plugin-based command line mechanismWhile the `beat` commandline is nice, @andre.anjos pointed out that a plugin-based commandline would be more useful.
I could not find an issue for this, so I'm creating this but feel free to close it if it is not relevant.
This was care...While the `beat` commandline is nice, @andre.anjos pointed out that a plugin-based commandline would be more useful.
I could not find an issue for this, so I'm creating this but feel free to close it if it is not relevant.
This was carefully designed and implemented in bob:
* https://gitlab.idiap.ch/bob/bob.extension/issues/44
* https://gitlab.idiap.ch/bob/bob.extension/merge_requests/64
* https://www.idiap.ch/software/bob/docs/bob/docs/master/bob.extension/doc/framework.html#unified-command-line-mechanism
A similar approach could be taken here as well.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/28Local plotting doesn't work2018-05-16T15:43:50ZJaden DIEFENBAUGHLocal plotting doesn't workDidn't have time to try the new plotting code until now...and plotting local experiments doesn't work.
I try to plot an experiment (`beat exp plot test/tutorial/eigenface/1/atnt`) and I get this:
```
retrieving `plotters/plotterparamete...Didn't have time to try the new plotting code until now...and plotting local experiments doesn't work.
I try to plot an experiment (`beat exp plot test/tutorial/eigenface/1/atnt`) and I get this:
```
retrieving `plotters/plotterparameters/plot/isoroc/1'...
findfont: Found a missing font file. Rebuilding cache.
font search path ['/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/matplotlib/mpl-data/fonts/ttf', '/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/matplotlib/mpl-data/fonts/afm', '/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/matplotlib/mpl-data/fonts/pdfcorefonts']
Could not open font file /usr/share/fonts/noto/NotoColorEmoji.ttf
generated new fontManager
Traceback (most recent call last):
File "/home/jae/miniconda3/envs/beat/bin/beat", line 11, in <module>
sys.exit(main())
File "/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/beat/cmdline/scripts/beat.py", line 183, in main
return module.process(args)
File "/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/beat/cmdline/experiments.py", line 620, in process
force, 0, {}, args['--outputfolder'])
File "/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/beat/cmdline/experiments.py", line 550, in plot
data.as_dict()[the_data], output_name, None, indentation + 2, format_cache)
File "/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/beat/cmdline/plotters.py", line 257, in plot
fig = runnable.process(data_to_plot)
File "/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/beat/core/plotter.py", line 95, in process
return loader.run(self.obj, 'process', self.exc, inputs)
File "/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/beat/backend/python/loader.py", line 109, in run
return getattr(obj, method)(*args, **kwargs)
File "/home/jae/code/beat.editor/prefix/plotters/plot/isoroc/1.py", line 143, in process
return super(Plotter, self).encode_figure(fig)
File "/home/jae/code/beat.editor/prefix/libraries/plot/baselib/1.py", line 85, in encode_figure
if self.content_type.endswith('png'): canvas.print_png(sio)
File "/home/jae/miniconda3/envs/beat/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 528, in print_png
self.figure.dpi, metadata=metadata)
TypeError: string argument expected, got 'bytes'
```
Maybe the `plot/baselib/1` library should be updated?https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/27[all packages] code cleanup2018-09-04T09:09:56ZFlavio TARSETTI[all packages] code cleanupbeat.cmdline needs to get some things changed such as:
- type keyword to remove
- status = None returned (to return a real status)beat.cmdline needs to get some things changed such as:
- type keyword to remove
- status = None returned (to return a real status)https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/26[config] set2018-05-05T22:52:16ZFlavio TARSETTI[config] set@andre.anjos @jdiefenbaugh @samuel.gaist
There's a bug with the config set [options]
it systematically adds a "}" at the end of the local config file or sometimes a new user: "user"} line.
And it breaks everything afterwards.
Needs t...@andre.anjos @jdiefenbaugh @samuel.gaist
There's a bug with the config set [options]
it systematically adds a "}" at the end of the local config file or sometimes a new user: "user"} line.
And it breaks everything afterwards.
Needs to be checked.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/24Store prefix on configuration file2018-12-12T09:14:17ZAndré AnjosStore prefix on configuration fileIt would be nice to be able to store the prefix on the configuration file.It would be nice to be able to store the prefix on the configuration file.André AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/23[cache] experiments data does not contain all information such as "type" field2018-05-01T09:09:50ZFlavio TARSETTI[cache] experiments data does not contain all information such as "type" fieldWhen the experiments is run locally,
We have a major issue.
Why aren't the information saved the same way as in the API?
One of the most important information is missing in the cache.
I was expecting this as in the API:
"results...When the experiments is run locally,
We have a major issue.
Why aren't the information saved the same way as in the API?
One of the most important information is missing in the cache.
I was expecting this as in the API:
"results": {
"analysis": {
"bar_plot": {
"type": "plot/bar/1",
"primary": false,
"value": {
"data": [
{
"y": [
0.0,
1.0,
2.0,
But I was able to get only this from the local cache:
"bar_plot": {
"data": [
{
"y": [
0.0,
1.0,
2.0,
3.0,
Where is the "type" field gone?
The type field is important as it indicates which plotter can handle this kind of data.
At present, I am working on plotting using data I get remotely from the experiment when hitting the API, but someone would want to be able to use experiments data he has ran locally and this is not possible at the moment due to the "cache" missing information. This is blocking for local plotting from locally ran experiments.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/22[CI] issue with docker and zc.buildout2018-04-19T14:08:51ZFlavio TARSETTI[CI] issue with docker and zc.buildoutSeems like the CI is having a strange issue with docker and and zc.buildout (apparently):
https://gitlab.idiap.ch/beat/beat.cmdline/-/jobs/132525
No hurry @samuel.gaist, it's not blocking for me at the moment, it was just to raise your...Seems like the CI is having a strange issue with docker and and zc.buildout (apparently):
https://gitlab.idiap.ch/beat/beat.cmdline/-/jobs/132525
No hurry @samuel.gaist, it's not blocking for me at the moment, it was just to raise your attention on this.
Thanks,
Flaviohttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/21Add local environment querying support2018-05-15T13:53:51ZJaden DIEFENBAUGHAdd local environment querying support(This is the first part of beat.editor#77)
Right now `beat.core` can tell us about the available local environments available (see [`dock.py`](https://gitlab.idiap.ch/beat/beat.core/blob/a8871304860b9e7c87f3b7ec08c0b308ccd0908a/beat/cor...(This is the first part of beat.editor#77)
Right now `beat.core` can tell us about the available local environments available (see [`dock.py`](https://gitlab.idiap.ch/beat/beat.core/blob/a8871304860b9e7c87f3b7ec08c0b308ccd0908a/beat/core/dock.py#L149)). The goal is to give `beat.editor` access to this info, which means that `beat.cmdline` needs a nice way to supply this info.
I'd add a new beat sub-app, `environment` (with an `env` shortcut) with a `list` command that would give the necessary info. Ideally the environment info would be provided in some machine-parseable format and provide at least:
- environment names & versions
- queues for each environment along with their slot counts
- the environments' packages (names & versions)
Right now I'm thinking to just use JSON of the same format as the beat.web endpoint for environments. [BEAT staging example](http://beatweb-staging.lab.idiap.ch:80/api/v1/backend/environments/):
```json
[
{
"name": "Scientific Python 2.7",
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "0.0.4",
"short_description": "Scientific Python 2.7"
},
{
"name": "Scientific Python 2.7",
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "1.0.0",
"short_description": "Scientific Python 2.7"
}
]
```
The one thing missing is the package info. There would be a `"packages"` field added to each environment that points to a dict consisting of package names for keys and versions for values:
```json
[
{
"name": "Scientific Python 2.7",
"packages": {
"alabaster": "0.7.6",
"babel": "1.3",
...
},
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "0.0.4",
"short_description": "Scientific Python 2.7"
},
{
"name": "Scientific Python 2.7",
"packages": {
"alabaster": "0.7.10",
"Babel": "2.4.0",
...
},
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "1.0.0",
"short_description": "Scientific Python 2.7"
}
]
```
This is the same extension proposed in beat.web#495.Jaden DIEFENBAUGHJaden DIEFENBAUGHhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/20Format for storing plotterparameters in prefix2018-05-15T13:54:13ZJaden DIEFENBAUGHFormat for storing plotterparameters in prefix@flavio.tarsetti how does the system associate plotters & plotterparameters?
The names aren't associated, as the plotterparameter `plot/det_40x40/1` isn't a similar name to `plot/isoroc/1`. And the plotterparameter content is:
```json
...@flavio.tarsetti how does the system associate plotters & plotterparameters?
The names aren't associated, as the plotterparameter `plot/det_40x40/1` isn't a similar name to `plot/isoroc/1`. And the plotterparameter content is:
```json
{
"grid": true,
"title": "DET ISO/IEC 19795-1:2006",
"title-fontsize": 12,
"det": true,
"legend-loc": "best",
"width": 800,
"height": 600,
"dpi": 120,
"legend-fontsize": 8,
"xlim-left": 0,
"xlim-right": 40,
"ylim-bottom": 0,
"ylim-top": 40,
"axis-fontsize": 10
}
```
How do we associate them?https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/19Fetch environment data from beat.web instances2018-04-13T12:39:53ZJaden DIEFENBAUGHFetch environment data from beat.web instances...using the `/api/v1/backend/environments/` endpoint.
Not sure how this would be stored...probably just dump the JSON in an `environments.json` file in the `.beat` user folder.
Needed for the June 2018 release!...using the `/api/v1/backend/environments/` endpoint.
Not sure how this would be stored...probably just dump the JSON in an `environments.json` file in the `.beat` user folder.
Needed for the June 2018 release!https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/18Autocompletion of BEAT objects in prefix & commands2018-03-23T08:38:40ZJaden DIEFENBAUGHAutocompletion of BEAT objects in prefix & commandsThis would improve productivity SO MUCH. I almost think this is a required feature for the June 2018 deliverable. We'd probably want to pull in a new dependency...maybe even need to rewrite a bunch of the CLI code. But I'm not sure since...This would improve productivity SO MUCH. I almost think this is a required feature for the June 2018 deliverable. We'd probably want to pull in a new dependency...maybe even need to rewrite a bunch of the CLI code. But I'm not sure since I haven't worked much with CLI python.