beat issues
https://gitlab.idiap.ch/groups/beat/-/issues
2020-10-19T15:16:45Z
https://gitlab.idiap.ch/beat/beat.core/-/issues/74
Update the schemas to better reflect parameter type support in different objects
2020-10-19T15:16:45Z
Jaden DIEFENBAUGH
Update the schemas to better reflect parameter type support in different objects
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algor...
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algorithm schema, `algorithm/common.json:80`:
```json
80 "parameter": {
81 "type": "object",
82 "properties": {
83 "type": { "$ref": "../common/1.json#/definitions/basetype" },
84 "default": { "$ref": "../common/1.json#/definitions/value" },
85 "description": { "type": "string" }
86 },
87 "required": [
88 "type"
89 ]
90 },
```
The `type` field references the `basetype` types, which includes complex types, but the editors (`beat.web` & `beat.editor`) don't actually support complex types for algorithm parameters.
I'll be looking through the other schemas and will update this issue with other things I find, if any.
EDIT: didn't find anything else in the other objects' schemas parameter definitions
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.editor/-/issues/205
Algorithm Editor Parameter: unknown types "array" and "dict"
2019-09-27T11:58:26Z
Flavio TARSETTI
Algorithm Editor Parameter: unknown types "array" and "dict"
@jdiefenbaugh : from where does these 2 types "array" and "dict" come from ?
As can be seen here from the `basetype` field in `beat/core/schema/common/1.json` these 2 types are non existant in any of these schema, which makes me wonder w...
@jdiefenbaugh : from where does these 2 types "array" and "dict" come from ?
As can be seen here from the `basetype` field in `beat/core/schema/common/1.json` these 2 types are non existant in any of these schema, which makes me wonder where did you get them from ?
As it should always follow a specific schema from beat.core
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.core/-/issues/73
simplejson VS python json
2019-03-20T11:27:56Z
Samuel GAIST
simplejson VS python json
There's currently a mix of use between the python official json module and simplejson.
One of the side effect seen is that the official json module handles less input types as simplejson until python 3.6.
This can be an issue when usin...
There's currently a mix of use between the python official json module and simplejson.
One of the side effect seen is that the official json module handles less input types as simplejson until python 3.6.
This can be an issue when using the system provided Python 3 on Debian 9 which is currently 3.5
Example of such situation:
- Receiving data through the network -> bytes are returned in Python 3
- Loading data from disk using pkg_resources -> bytes are returned
From a backward compatible point of view, it would make more sense to use simplejson because of these use cases.
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/72
Assert usage cleanup
2019-05-09T13:26:49Z
Samuel GAIST
Assert usage cleanup
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.core.
Depending on how they are used, the following actions can be done:
1. Turn the statement into ...
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.core.
Depending on how they are used, the following actions can be done:
1. Turn the statement into an exception
2. Handle the case gracefully
3. Mark as not to get worried about
The current state of the code base is usually 3 which should be cleaned up.
All tests in this repository have been cleaned to use the testing framework corresponding tools
https://gitlab.idiap.ch/beat/beat.core/-/issues/71
Algorithm prototype loading fails on python 3.5
2019-03-15T10:41:57Z
Samuel GAIST
Algorithm prototype loading fails on python 3.5
Issue similar to the one fixed by !58
`pkg_resources.resource_string` returns bytes and not str which the json module doesn't load on 3.5.
Issue found through beat/beat.web>
Issue similar to the one fixed by !58
`pkg_resources.resource_string` returns bytes and not str which the json module doesn't load on 3.5.
Issue found through beat/beat.web>
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/70
ZMQ architecture documentation
2019-03-14T14:32:54Z
Samuel GAIST
ZMQ architecture documentation
This issue is for tracking the documentation of the ZMQ architecture.
This issue is for tracking the documentation of the ZMQ architecture.
ZMQ resilience improvement
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.editor/-/issues/204
[reusable-component] Library Editor
2019-10-04T15:12:53Z
Flavio TARSETTI
[reusable-component] Library Editor
A reusable component for library selection is required for various editors.
A reusable component for library selection is required for various editors.
Flavio TARSETTI
Flavio TARSETTI
https://gitlab.idiap.ch/beat/beat.web/-/issues/520
ZMQ refactor deployment
2019-03-14T10:34:40Z
Samuel GAIST
ZMQ refactor deployment
This issue tracks the ZMQ deployment
* [x] Staging
* [x] Production
This issue tracks the ZMQ deployment
* [x] Staging
* [x] Production
Scheduler/Worker resilience improvement
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.editor/-/issues/203
MR pipeline
2019-03-09T08:19:14Z
Flavio TARSETTI
MR pipeline
@andre.anjos :
after the fix of bdt with the numpy pinning issue https://gitlab.idiap.ch/bob/bob.conda/merge_requests/403:
Please make sure to re-run this MR pipeline please:
https://gitlab.idiap.ch/beat/beat.editor/pipelines/27862
Thanks
@andre.anjos :
after the fix of bdt with the numpy pinning issue https://gitlab.idiap.ch/bob/bob.conda/merge_requests/403:
Please make sure to re-run this MR pipeline please:
https://gitlab.idiap.ch/beat/beat.editor/pipelines/27862
Thanks
André Anjos
André Anjos
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/57
Not compatible with newest bob-devel version
2019-03-09T09:20:51Z
Jaden DIEFENBAUGH
Not compatible with newest bob-devel version
See beat.editor#202 for how I found this & a hack to fix it
Using the default conda environment creation config, `beat.cmdline` can't be installed to the same environment as `bob-devel=2019.2.11`. This also pops up when trying to create...
See beat.editor#202 for how I found this & a hack to fix it
Using the default conda environment creation config, `beat.cmdline` can't be installed to the same environment as `bob-devel=2019.2.11`. This also pops up when trying to create an environment for `beat.web` using `conda env create -f dev.yml`:
```
Collecting package metadata: done
Solving environment: failed
UnsatisfiableError: The following specifications were found to be in conflict:
- beat.cmdline
- bob-devel=2019.02.11
Use "conda search <package> --info" to see the dependencies for each package.
```
This marks the third place this issue has shown up, and so I'm opening an issue in this package. I'll mark it as `critical` because this is an issue in the CI as well.
https://gitlab.idiap.ch/beat/beat.editor/-/issues/202
`bdt create` can't reconcile beat.cmdline with specified devel packages
2019-03-09T09:22:14Z
Jaden DIEFENBAUGH
`bdt create` can't reconcile beat.cmdline with specified devel packages
Running `bdt create -vv --python=3.6 beat_editor` on master will fail, conda unable to find compatible `beat.cmdline` & `bob-devel` versions:
```
Solving environment: ...working... failed
UnsatisfiableError: The following specifications...
Running `bdt create -vv --python=3.6 beat_editor` on master will fail, conda unable to find compatible `beat.cmdline` & `bob-devel` versions:
```
Solving environment: ...working... failed
UnsatisfiableError: The following specifications were found to be in conflict:
- beat.cmdline
- bob-devel=2019.02.11
Use "conda search <package> --info" to see the dependencies for each package.
Traceback (most recent call last):
File "/home/jae/miniconda3/envs/bdt/bin/bdt", line 11, in <module>
sys.exit(main())
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/bob/devtools/scripts/bdt.py", line 42, in _decorator
value = view_func(*args, **kwargs)
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/bob/devtools/scripts/create.py", line 145, in create
dry_run, use_local)
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/bob/devtools/build.py", line 285, in conda_create
run_cmdline(cmd)
File "/home/jae/miniconda3/envs/bdt/lib/python3.6/site-packages/bob/devtools/bootstrap.py", line 118, in run_cmdline
(' '.join(cmd), p.returncode))
RuntimeError: command `/home/jae/miniconda3/bin/conda create --yes --name beat_editor_2 --override-channels --channel=http://www.idiap.ch/software/bob/conda/label/beta --channel=http://www.idiap.ch/software/bob/conda --channel=defaults beat-devel=2019.02.11.* beat.cmdline bob-devel=2019.02.11.* bob.buildout bob.extension ca-certificates certifi coverage docopt flask flask-cors flask-restful ipdb jinja2 libedit libffi libgcc libgcc-ng libstdcxx-ng mr.developer ncurses nodejs nodejs nose openssl pip python python>=3.6,<3.7.0a0 readline setuptools setuptools simplejson sphinx sphinx_rtd_theme sqlite tk wheel xz zlib' exited with error state (1)
```
I'm not sure if this is a `beat.editor`, `beat.cmdline`, or `bob.devtools` issue, so I'm opening it here first since this is where I encountered it.
After messing around with everything, I found a hack to fix this - change the pinned versions of `bob_devel` and `beat_devel` in `miniconda3/envs/bdt/lib/python3.6/site-packages/bob/devtools/data/conda_build_config.yaml:52` to use versions released in 2018:
```yaml
# This version of bob-devel will be used at test time of packages:
bob_devel:
- 2018.*
#2019.02.11
# This version of beat-devel will be used at test time of packages. Notice it
# uses bob-devel and should have a version that is greater or equal its value
beat_devel:
- 2018.*
#2019.02.11
```
Hopefully this is just a conda config issue for `beat.editor` or `beat.cmdline`, but maybe we need a custom `config.yaml` to pass to the `--config` flag in `bdt create`?
(I encountered this error when working on #201)
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.editor/-/issues/201
[doc] development
2019-03-12T11:34:22Z
Flavio TARSETTI
[doc] development
@jdiefenbaugh
I broke my conda environment and need to reinstall beat.editor v1 and followed
`https://gitlab.idiap.ch/beat/beat.editor/blob/master/doc/development.rst`
However given that:
```python
$ ../bob.admin/conda/conda-bootstra...
@jdiefenbaugh
I broke my conda environment and need to reinstall beat.editor v1 and followed
`https://gitlab.idiap.ch/beat/beat.editor/blob/master/doc/development.rst`
However given that:
```python
$ ../bob.admin/conda/conda-bootstrap.py --overwrite --python=3.6 beatedit
```
is not valid anymore, can you update the new procedure in the documentation `https://gitlab.idiap.ch/beat/beat.editor/blob/master/doc/development.rst` ?
Thanks!
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.core/-/issues/69
Errors building with bob.devtools
2019-02-28T07:39:40Z
Jaden DIEFENBAUGH
Errors building with bob.devtools
I need to build it to work on the plotterparameter schema but it didn't work. I ran
```
$ bdt build -vv . --python=3.6
```
The first errors are the following:
```
...
running install_egg_info
Copying beat.core.egg-info to $PREFIX/lib/pyt...
I need to build it to work on the plotterparameter schema but it didn't work. I ran
```
$ bdt build -vv . --python=3.6
```
The first errors are the following:
```
...
running install_egg_info
Copying beat.core.egg-info to $PREFIX/lib/python3.6/site-packages/beat.core-1.8.0b0-py3.6.egg-info
File "/home/jae/miniconda3/conda-bld/beat.core_1551291776091/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac
ehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/beat/core/test/prefix/databases/invalid/1.py", line 28
class View; # <-- syntax error!
^
SyntaxError: invalid syntax
File "/home/jae/miniconda3/conda-bld/beat.core_1551291776091/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac
ehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/beat/core/test/prefix/algorithms/user/syntax_error/1.py"
, line 29
class Algorithm;
^
SyntaxError: invalid syntax
...
```
Hopefully these errors aren't blocking the build because those syntax errors are actually intentional! The build continues and gets conda `ClobberWarning`s:
```
...
ERROR:conda.stderr:
ClobberWarning: This transaction has incompatible packages due to a shared path.
packages: http://www.idiap.ch/software/bob/conda/label/beta::beat.backend.python-1.6.2b0-py36h3c37dc0_3, file:///home/jae/miniconda3/conda-bld::beat.core-0.0.1-py36h1c6dc61_0
path: 'lib/python3.6/site-packages/beat/__init__.py'
...
```
After that there's alot of errors and exceptions, so I'm attaching the [bdt build log](/uploads/872db60d37880e72435fbc1020201099/bdt_build.log) and my [bdt environment package list](/uploads/32b39df45c96ab133890e5f63944f0c6/bdt_env_list.log).
This is using a python 3.6 conda environment and specifying python 3.6 for building beat.core, but the same-ish thing happens using python 3.7.
Should I not be using bob.devtools to build beat.core yet, or am I missing something?
https://gitlab.idiap.ch/beat/beat.editor/-/issues/200
Package.json became package.json.template
2019-02-27T19:58:34Z
Jaden DIEFENBAUGH
Package.json became package.json.template
What happened? Renaming `package.json` to `package.json.template` breaks *alot* of stuff as everything nodejs-based assumes there's a file called `package.json` at the package root.
What happened? Renaming `package.json` to `package.json.template` breaks *alot* of stuff as everything nodejs-based assumes there's a file called `package.json` at the package root.
https://gitlab.idiap.ch/beat/beat.web/-/issues/518
Scheduling refactor
2019-03-11T13:42:40Z
Samuel GAIST
Scheduling refactor
beat/beat.core> has been refactored to improve the reliability of the ZMQ.
This task tracks the work related to update the scheduling process to use that new architecture.
beat/beat.core> has been refactored to improve the reliability of the ZMQ.
This task tracks the work related to update the scheduling process to use that new architecture.
Scheduler/Worker resilience improvement
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/68
Refactor ZMQ architecture
2019-03-13T07:55:39Z
Samuel GAIST
Refactor ZMQ architecture
The current architecture works well but connection with the scheduler may get lost after a long period of network inactivity.
The task here is to refactor the architecture so that adding and removing node as well as crashing nodes gets ...
The current architecture works well but connection with the scheduler may get lost after a long period of network inactivity.
The task here is to refactor the architecture so that adding and removing node as well as crashing nodes gets improved handling and thus avoids as much as possible "blocked" experiments.
ZMQ resilience improvement
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.web/-/issues/517
Docs/usage on scheduler/worker need to be updated
2019-03-11T13:42:40Z
Jaden DIEFENBAUGH
Docs/usage on scheduler/worker need to be updated
A couple things:
1. The admin docs (`doc/admin/installation.rst`) says to run `./bin/scheduler -vv` and `./bin/worker -vv` to start a single scheduler/worker node. This doesn't work anymore and needs to be updated.
2. Running `./bin/sche...
A couple things:
1. The admin docs (`doc/admin/installation.rst`) says to run `./bin/scheduler -vv` and `./bin/worker -vv` to start a single scheduler/worker node. This doesn't work anymore and needs to be updated.
2. Running `./bin/scheduler --help` gives the following message after the usage/options sections:
```
Examples:
To start the scheduling process do the following:
$ scheduler
You can pass the ``-v`` flag to start the scheduler with the logging level
set to ``INFO`` or ``-vv`` to set it to ``DEBUG``. By default, the logging
level is set to ``WARNING`` if no ``-v`` flag is passed.
```
This isn't correct either.
Running `./bin/scheduler` gives the following error:
```
Traceback (most recent call last):
File "./bin/scheduler", line 26, in <module>
sys.exit(beat.web.scripts.scheduler.main())
File "/home/jae/work/beat.web/beat/web/scripts/scheduler.py", line 219, in main
int(arguments['--port']),
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
```
I tried providing a valid value to the `--port` argument: `./bin/scheduler --port=9101`, but it gave me a ZMQ error:
```
Traceback (most recent call last):
File "./bin/scheduler", line 26, in <module>
sys.exit(beat.web.scripts.scheduler.main())
File "/home/jae/work/beat.web/beat/web/scripts/scheduler.py", line 222, in main
onWorkerGone = onWorkerGone,
File "/home/jae/miniconda3/envs/beat-web/lib/python3.6/site-packages/beat/core/worker.py", line 99, in __init__
self.socket.bind(self.address)
File "zmq/backend/cython/socket.pyx", line 547, in zmq.backend.cython.socket.Socket.bind
File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: No such device
```
Trying to start a worker node via `./bin/worker` gives the following message:
```
Usage:
worker [-v ... | --verbose ...] [ --name=<name>] [--prefix=<path>]
[--cache=<path>] [--docker] [--docker-network=<name>]
[--port-range=<range>] <scheduler_address>
worker (--help | -h)
worker (--version | -V)
```
I did all this while `./bin/django runserver` was running in a different window.
This is a blocker on some of my work as I need to run experiments successfully to create reports to work on the report-related issues.
Scheduler/Worker resilience improvement
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/67
Algorithm prototype default dataformat not available on platform installation
2019-02-25T08:32:27Z
Samuel GAIST
Algorithm prototype default dataformat not available on platform installation
The `integers` dataformat used as default type when loading the algorithm prototype is not available on a default installation of the BEAT platoform.
The `integers` dataformat used as default type when loading the algorithm prototype is not available on a default installation of the BEAT platoform.
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/66
The plotterparameter type doesn't have a JSON schema file
2019-04-18T22:25:57Z
Jaden DIEFENBAUGH
The plotterparameter type doesn't have a JSON schema file
Not sure what happened but we need one, right?
I found this out by working on https://gitlab.idiap.ch/beat/beat.editor/issues/161
Not sure what happened but we need one, right?
I found this out by working on https://gitlab.idiap.ch/beat/beat.editor/issues/161
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.web/-/issues/516
Issues with installing demo data on v1.4.1 and master
2019-02-25T18:34:53Z
Jaden DIEFENBAUGH
Issues with installing demo data on v1.4.1 and master
When installing the dummy data in a new installation via `./bin/django install`, all the experiments & algorithms aren't installed due to errors (log attached). As I'm working on the reports app, I need at least some data to make reports...
When installing the dummy data in a new installation via `./bin/django install`, all the experiments & algorithms aren't installed due to errors (log attached). As I'm working on the reports app, I need at least some data to make reports with.
I tried alot of different things hoping that I was doing something wrong, but I think the test data just hasn't been updated?
Repro:
- Clone beat.web
- Either stay on master or checkout tag `v.1.4.1`
- Run the commands in the README to install beat.web through conda/buildout
- Run `./bin/django install -v1` or `./bin/django install`
Expected:
The example data is installed to the new platform installation successfully and is available when running `./bin/django runserver`.
Actual:
There are only 14 toolchains, 2 libraries, and 10 dataformats available. Nothing else.
Logs:
Log for `./bin/django install`: [install.out](/uploads/8240025bc185d6aeca7d4a6072dafe27/install.out)
Log for `./bin/django install -v1`: [install_v1.out](/uploads/b91feb30aae43465e4a57982e50882d2/install_v1.out)