beat.cmdline issueshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues2018-12-06T09:50:13Zhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/52About Walkthrough section in the documentation.2018-12-06T09:50:13ZZohreh MOSTAANIAbout Walkthrough section in the documentation.I do not understand what is the purpose of the section Walkthrough in the documentation of beat.cmdline. Is it for developing the beat platform locally? Is it only for running the platform locally? Or is it for developing the beat.cmdlin...I do not understand what is the purpose of the section Walkthrough in the documentation of beat.cmdline. Is it for developing the beat platform locally? Is it only for running the platform locally? Or is it for developing the beat.cmdline itself? In the first line of this documentation it is written:
This page gives an overview on setting up an environment to develop BEAT **experiments** locally.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/58Add BEAT classifier to setup.py2019-04-11T08:58:40ZSamuel GAISTAdd BEAT classifier to setup.pyThe classifier has been added to Pypi so it now can be used.
See beat/beat.backend.python#19The classifier has been added to Pypi so it now can be used.
See beat/beat.backend.python#19https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/16Add experiment results plot generation command2018-05-15T13:54:56ZJaden DIEFENBAUGHAdd experiment results plot generation commandI don't see a way to generate~visualize results from successfully-ran experiments outside of pushing the experiment to a `beat.web` installation. This feature seems practically necessary for a easy development cycle.
Maybe add a sub-com...I don't see a way to generate~visualize results from successfully-ran experiments outside of pushing the experiment to a `beat.web` installation. This feature seems practically necessary for a easy development cycle.
Maybe add a sub-command to the experiments or plotters command groups?Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/21Add local environment querying support2018-05-15T13:53:51ZJaden DIEFENBAUGHAdd local environment querying support(This is the first part of beat.editor#77)
Right now `beat.core` can tell us about the available local environments available (see [`dock.py`](https://gitlab.idiap.ch/beat/beat.core/blob/a8871304860b9e7c87f3b7ec08c0b308ccd0908a/beat/cor...(This is the first part of beat.editor#77)
Right now `beat.core` can tell us about the available local environments available (see [`dock.py`](https://gitlab.idiap.ch/beat/beat.core/blob/a8871304860b9e7c87f3b7ec08c0b308ccd0908a/beat/core/dock.py#L149)). The goal is to give `beat.editor` access to this info, which means that `beat.cmdline` needs a nice way to supply this info.
I'd add a new beat sub-app, `environment` (with an `env` shortcut) with a `list` command that would give the necessary info. Ideally the environment info would be provided in some machine-parseable format and provide at least:
- environment names & versions
- queues for each environment along with their slot counts
- the environments' packages (names & versions)
Right now I'm thinking to just use JSON of the same format as the beat.web endpoint for environments. [BEAT staging example](http://beatweb-staging.lab.idiap.ch:80/api/v1/backend/environments/):
```json
[
{
"name": "Scientific Python 2.7",
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "0.0.4",
"short_description": "Scientific Python 2.7"
},
{
"name": "Scientific Python 2.7",
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "1.0.0",
"short_description": "Scientific Python 2.7"
}
]
```
The one thing missing is the package info. There would be a `"packages"` field added to each environment that points to a dict consisting of package names for keys and versions for values:
```json
[
{
"name": "Scientific Python 2.7",
"packages": {
"alabaster": "0.7.6",
"babel": "1.3",
...
},
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "0.0.4",
"short_description": "Scientific Python 2.7"
},
{
"name": "Scientific Python 2.7",
"packages": {
"alabaster": "0.7.10",
"Babel": "2.4.0",
...
},
"queues": {
"Default": {
"memory_limit": 5983,
"nb_slots": 2,
"max_slots_per_user": 2,
"nb_cores_per_slot": 1,
"time_limit": 360
}
},
"accessibility": "public",
"languages": [
"python"
],
"version": "1.0.0",
"short_description": "Scientific Python 2.7"
}
]
```
This is the same extension proposed in beat.web#495.Jaden DIEFENBAUGHJaden DIEFENBAUGHhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/62Add support for remote experiments execution handling2019-09-16T17:30:12ZSamuel GAISTAdd support for remote experiments execution handlingCurrently, beat/beat.cmdline> only offers to run experiment locally.
This issue tracks the implementation of the commands needed to start, cancel and monitor a remote experiment.
The current monitoring implementation in beat/beat.web> ...Currently, beat/beat.cmdline> only offers to run experiment locally.
This issue tracks the implementation of the commands needed to start, cancel and monitor a remote experiment.
The current monitoring implementation in beat/beat.web> is based on a 5 second timer which, on timeout, will fetch the running experiment and, based on the content returned, update the information panel and toolchain representation.Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/34`beat caches view` doesn't work2018-12-12T09:14:17ZJaden DIEFENBAUGH`beat caches view` doesn't workI get the following:
```
path: e4/fb/dc/b2f4751a65179e51b472c4b8c1efbec86441e61194bc02442971ae3405
dataformat: tutorial/linear_machine/1
Traceback (most recent call last):
File "/home/jae/miniconda3/envs/beatedit/bin/beat", line 11, ...I get the following:
```
path: e4/fb/dc/b2f4751a65179e51b472c4b8c1efbec86441e61194bc02442971ae3405
dataformat: tutorial/linear_machine/1
Traceback (most recent call last):
File "/home/jae/miniconda3/envs/beatedit/bin/beat", line 11, in <module>
sys.exit(main())
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/cmdline/scripts/beat.py", line 186, in main
return module.process(args)
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/cmdline/cache.py", line 191, in process
return view(args['config'], args['<path>'], start, end)
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/cmdline/cache.py", line 164, in view
cls=NumpyJSONEncoder).\
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/__init__.py", line 397, in dumps
**kw).encode(obj)
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 293, in encode
chunks = list(chunks)
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 677, in _iterencode
for chunk in _iterencode_dict(o, _current_indent_level):
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 634, in _iterencode_dict
for chunk in chunks:
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 697, in _iterencode
o = _default(o)
File "/home/jae/miniconda3/envs/beatedit/lib/python3.6/site-packages/beat/backend/python/utils.py", line 354, in default
return simplejson.JSONEncoder.default(self, obj)
File "/home/jae/.local/lib/python3.6/site-packages/simplejson/encoder.py", line 268, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <map object at 0x7f1b3d54a4e0> is not JSON serializable
```
This happens with any cache I've tried. Does it work for anyone?Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/55beat --version does not work at latest release2019-01-21T14:07:25ZAndré Anjosbeat --version does not work at latest releaseWhat I get:
```sh
$ beat --version
Usage: ....
Error: Missing Command.
$
```What I get:
```sh
$ beat --version
Usage: ....
Error: Missing Command.
$
```Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/43cache should be cleared to run new experiments even if the changes are in the...2018-08-18T04:34:25ZZohreh MOSTAANIcache should be cleared to run new experiments even if the changes are in the database block.from What I understood from the documentation of beat.editor about cache system, it seems that if you change something in a block, beat should rerun that block and all the blocks depending on it. But it is not the case.
I changed the ro...from What I understood from the documentation of beat.editor about cache system, it seems that if you change something in a block, beat should rerun that block and all the blocks depending on it. But it is not the case.
I changed the root folder for the database atnt/5 in the prefix given in beat.tutorial.prefix to run the experiment test/tutorial/eigenface/1/atnt , but I had to clear the cache, otherwise it would still point to the wrong address for the database, even-though the json file was updated immediately.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/73Cannot push a C++ algorithm2020-07-23T12:24:14ZAmir MOHAMMADICannot push a C++ algorithmLooks like when pushing a C++ algorithm, the code fails with:
```
lib/python3.7/site-packages/click/core.py(829)__call__()
828 """Alias for :meth:`main`."""
--> 829 return self.main(*args, **kwargs)
830
lib/...Looks like when pushing a C++ algorithm, the code fails with:
```
lib/python3.7/site-packages/click/core.py(829)__call__()
828 """Alias for :meth:`main`."""
--> 829 return self.main(*args, **kwargs)
830
lib/python3.7/site-packages/click/core.py(782)main()
781 with self.make_context(prog_name, args, **extra) as ctx:
--> 782 rv = self.invoke(ctx)
783 if not standalone_mode:
lib/python3.7/site-packages/click/core.py(1259)invoke()
1258 with sub_ctx:
-> 1259 return _process_result(sub_ctx.command.invoke(sub_ctx))
1260
lib/python3.7/site-packages/click/core.py(1259)invoke()
1258 with sub_ctx:
-> 1259 return _process_result(sub_ctx.command.invoke(sub_ctx))
1260
lib/python3.7/site-packages/beat/cmdline/click_helper.py(121)invoke()
119
120 ctx.meta["asset_info"] = self.asset_info
--> 121 return super().invoke(ctx)
lib/python3.7/site-packages/click/core.py(1066)invoke()
1065 if self.callback is not None:
-> 1066 return ctx.invoke(self.callback, **ctx.params)
1067
lib/python3.7/site-packages/click/core.py(610)invoke()
609 with ctx:
--> 610 return callback(*args, **kwargs)
611
lib/python3.7/site-packages/click/decorators.py(21)new_func()
20 def new_func(*args, **kwargs):
---> 21 return f(get_current_context(), *args, **kwargs)
22
lib/python3.7/site-packages/beat/cmdline/decorators.py(94)_decorator()
93 def _decorator(*args, **kwargs):
---> 94 value = view_func(*args, **kwargs)
95 if value not in [None, 0]:
lib/python3.7/site-packages/beat/cmdline/commands.py(321)push_impl()
320 dry_run=dry_run,
--> 321 indentation=0,
322 )
lib/python3.7/site-packages/beat/cmdline/common.py(1486)push()
1485 try:
-> 1486 webapi.post(url, data=message)
1487 except RuntimeError as e:
> lib/python3.7/site-packages/beat/cmdline/webapi.py(104)post()
103 url = self.__build_url(path)
--> 104 answer = requests.post(url, json=data, headers=self._make_headers())
105
lib/python3.7/site-packages/requests/api.py(119)post()
118
--> 119 return request('post', url, data=data, json=json, **kwargs)
120
lib/python3.7/site-packages/requests/api.py(61)request()
60 with sessions.Session() as session:
---> 61 return session.request(method=method, url=url, **kwargs)
62
lib/python3.7/site-packages/requests/sessions.py(516)request()
515 )
--> 516 prep = self.prepare_request(req)
517
lib/python3.7/site-packages/requests/sessions.py(459)prepare_request()
458 cookies=merged_cookies,
--> 459 hooks=merge_hooks(request.hooks, self.hooks),
460 )
lib/python3.7/site-packages/requests/models.py(317)prepare()
316 self.prepare_cookies(cookies)
--> 317 self.prepare_body(data, files, json)
318 self.prepare_auth(auth, url)
lib/python3.7/site-packages/requests/models.py(467)prepare_body()
466 content_type = 'application/json'
--> 467 body = complexjson.dumps(json)
468 if not isinstance(body, bytes):
lib/python3.7/site-packages/simplejson/__init__.py(395)dumps()
394 ):
--> 395 return _default_encoder.encode(obj)
396 if cls is None:
lib/python3.7/site-packages/simplejson/encoder.py(296)encode()
295 # equivalent to the PySequence_Fast that ''.join() would do.
--> 296 chunks = self.iterencode(o, _one_shot=True)
297 if not isinstance(chunks, (list, tuple)):
lib/python3.7/site-packages/simplejson/encoder.py(378)iterencode()
377 try:
--> 378 return _iterencode(o, 0)
379 finally:
```
I will not attach the binary the `.so` file (can be found in https://filesender.switch.ch/filesender/?vid=59e1dbc9-04e4-67c8-2bb1-00007d77b3bb but you don't need that)
Steps to reproduce:
* create an algorithm with this .json file [1.json](/uploads/d625814e021f43f4543060e5b3045b55/1.json)
* generate a random binary file for this alg and save it as `1.so` for example this: [1.so](/uploads/d7507a1afdff15ed1c07c1b6de9403d7/1.so)
* push it to the online platformSamuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/70Cannot specify plotterparmeters when calling `beat exp plot`2020-07-28T14:57:58ZAmir MOHAMMADICannot specify plotterparmeters when calling `beat exp plot`I was thinking that we can fix this by providing the plotterparams using click optiona and also loading from rc.I was thinking that we can fix this by providing the plotterparams using click optiona and also loading from rc.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/42cannot unassign an algorithm to a block in the experiment.2018-08-07T14:18:05ZZohreh MOSTAANIcannot unassign an algorithm to a block in the experiment.After choosing an algorithm for a block in an experiment it is not possible to assign no algorithm to that block. It is only possible to choose from certain algorithms. This is not the case for the Database though. If there is a problem ...After choosing an algorithm for a block in an experiment it is not possible to assign no algorithm to that block. It is only possible to choose from certain algorithms. This is not the case for the Database though. If there is a problem in the toolchain and you want to find out where the problem is it would be a good option to unassign an algorithm without deleting the whole block.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/47Change license to BSD 3-clause2019-03-13T14:44:15ZAndré AnjosChange license to BSD 3-clauseAs discussed in earlier weekly meetings (see action points from week02/2018 till week09/2018), the prospective re-use of BEAT core components to re-implement some Bob frameworks would require a shift in the licensing terms for some core ...As discussed in earlier weekly meetings (see action points from week02/2018 till week09/2018), the prospective re-use of BEAT core components to re-implement some Bob frameworks would require a shift in the licensing terms for some core BEAT packages.
@samuel.gaist: could you please do that *before* the release? The idea is to go from AGPLv3 to BSD 3-clause (as the rest of Bob) for all packages up to beat.cmdline. We exclude beat.web from this, leaving it as AGPLv3.
We should do the following, in details:
1. Remove the file `LICENSE.AGPL`
2. Create a new file called `LICENSE` the contains the stock BSD 3-clause text (see info here: https://gitlab.idiap.ch/bob/bob.admin/tree/master/templates)
3. Modify the file `MANIFEST.in` to include `LICENSE` instead of `LICENSE.AGPL`
4. Modify the file `setup.py` to list the license correctly on the classifier list (replace license key by `License :: OSI Approved :: BSD License`)
5. Modify the file `conda/meta.yaml` to set the license appropriately. Should read `license: BSD 3-Clause` and `license_family: BSD` on the `about` section, replacing the current AGPL stuff.
6. Double-check all headers from `.py` files - @flavio.tarsetti may have a script to do this.
Please consult with @philip.abbet since I think rechecked, by then, the feasibility of this move and found no issues against doing this.Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/48[ci] issue merge request2018-10-15T06:35:52ZFlavio TARSETTI[ci] issue merge requestOn this pipeline:
https://gitlab.idiap.ch/beat/beat.cmdline/pipelines/24234
One of the job output:
https://gitlab.idiap.ch/beat/beat.cmdline/-/jobs/150275
it fails all the time.
any idea why ? @zmostaani @samuel.gaist @andre.anjosOn this pipeline:
https://gitlab.idiap.ch/beat/beat.cmdline/pipelines/24234
One of the job output:
https://gitlab.idiap.ch/beat/beat.cmdline/-/jobs/150275
it fails all the time.
any idea why ? @zmostaani @samuel.gaist @andre.anjoshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/30[ci] issue with docker2018-05-10T12:13:54ZFlavio TARSETTI[ci] issue with dockerHi @andre.anjos ,
Do you see why docker is causing issues here ?
https://gitlab.idiap.ch/beat/beat.cmdline/pipelines/19968
ThanksHi @andre.anjos ,
Do you see why docker is causing issues here ?
https://gitlab.idiap.ch/beat/beat.cmdline/pipelines/19968
ThanksAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/22[CI] issue with docker and zc.buildout2018-04-19T14:08:51ZFlavio TARSETTI[CI] issue with docker and zc.buildoutSeems like the CI is having a strange issue with docker and and zc.buildout (apparently):
https://gitlab.idiap.ch/beat/beat.cmdline/-/jobs/132525
No hurry @samuel.gaist, it's not blocking for me at the moment, it was just to raise your...Seems like the CI is having a strange issue with docker and and zc.buildout (apparently):
https://gitlab.idiap.ch/beat/beat.cmdline/-/jobs/132525
No hurry @samuel.gaist, it's not blocking for me at the moment, it was just to raise your attention on this.
Thanks,
Flaviohttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/26[config] set2018-05-05T22:52:16ZFlavio TARSETTI[config] set@andre.anjos @jdiefenbaugh @samuel.gaist
There's a bug with the config set [options]
it systematically adds a "}" at the end of the local config file or sometimes a new user: "user"} line.
And it breaks everything afterwards.
Needs t...@andre.anjos @jdiefenbaugh @samuel.gaist
There's a bug with the config set [options]
it systematically adds a "}" at the end of the local config file or sometimes a new user: "user"} line.
And it breaks everything afterwards.
Needs to be checked.https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/77CXX algorithm cannot be correctly downloaded2020-11-10T16:42:45ZSamuel GAISTCXX algorithm cannot be correctly downloadedThe way the URL is built is wrong, it hardcodes the current use as the author which is not necessarily true.The way the URL is built is wrong, it hardcodes the current use as the author which is not necessarily true.Samuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/2Data (algorithms, dataformats, etc.) listing and pulling is not working any more2018-04-13T08:03:57ZLaurent EL SHAFEYData (algorithms, dataformats, etc.) listing and pulling is not working any moreData (algorithms, dataformats, etc.) listing and pulling is not working any more. The corresponding tests are failing as well.Data (algorithms, dataformats, etc.) listing and pulling is not working any more. The corresponding tests are failing as well.BTAS TutorialLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/3Data (algorithms, dataformats, etc.) pulling crashes when non-ascii character...2018-04-13T08:03:57ZLaurent EL SHAFEYData (algorithms, dataformats, etc.) pulling crashes when non-ascii characters are present in a fieldHere is an example:
```bash
lelshafey@futurix08:/idiap/user/lelshafey/work/beat_platform/beat.cmdline$ ./bin/beat algo pull
retrieving `algorithms/siebenkopf/GaborGraphSimilarity/12'...
Traceback (most recent call last):
File "....Here is an example:
```bash
lelshafey@futurix08:/idiap/user/lelshafey/work/beat_platform/beat.cmdline$ ./bin/beat algo pull
retrieving `algorithms/siebenkopf/GaborGraphSimilarity/12'...
Traceback (most recent call last):
File "./bin/beat", line 21, in <module>
sys.exit(beat.cmdline.scripts.beat.main())
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/scripts/beat.py", line 160, in main
return module.process(args)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/algorithms.py", line 144, in process
args['--force'], 0, {}, {})
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/algorithms.py", line 105, in pull
['declaration', 'code', 'description'], force, indentation)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/beat/cmdline/common.py", line 612, in pull
storage.save(**data)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/src/beat.core/beat/core/utils.py", line 262, in save
if description: self.doc.save(description)
File "/remote/idiap.svm/user.active/lelshafey/work/beat_platform/beat.cmdline/src/beat.core/beat/core/utils.py", line 129, in save
with open(self.path, mode) as f: f.write(contents)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 622: ordinal not in range(128)
```
This may be a problem with beat.core.BTAS TutorialLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.cmdline/-/issues/78Database index generated files belongs to docker daemon user2020-11-10T16:47:14ZSamuel GAISTDatabase index generated files belongs to docker daemon userWhich makes them belonging to root currently which is wrong.
It should be the user which currently runs the indexing or a specific UID.Which makes them belonging to root currently which is wrong.
It should be the user which currently runs the indexing or a specific UID.Samuel GAISTSamuel GAIST