beat issues
https://gitlab.idiap.ch/groups/beat/-/issues
2022-03-04T09:39:15Z
https://gitlab.idiap.ch/beat/beat.web/-/issues/577
Cleanup yearly supervision command
2022-03-04T09:39:15Z
Flavio TARSETTI
Cleanup yearly supervision command
The yearly supervison command needs a rewrite to improve clarity
The yearly supervison command needs a rewrite to improve clarity
Code cleanup
https://gitlab.idiap.ch/beat/beat.web/-/issues/553
toolchain created locally does not show up correctly on beat.web
2022-03-04T09:39:11Z
Amir MOHAMMADI
toolchain created locally does not show up correctly on beat.web
This toolchain [1.json](/uploads/49903eb5d5e5223a6f191a8656049ba8/1.json) also available here: https://www.idiap.ch/software/beat/platform/toolchains/amohammadi/livdet/1/#viewer does not show up correctly in the web platform but looks ok...
This toolchain [1.json](/uploads/49903eb5d5e5223a6f191a8656049ba8/1.json) also available here: https://www.idiap.ch/software/beat/platform/toolchains/amohammadi/livdet/1/#viewer does not show up correctly in the web platform but looks ok in beat.editor:
![image](/uploads/dfbfc0c76547e04de4db701dacb356c9/image.png)
https://gitlab.idiap.ch/beat/beat.editor/-/issues/274
Keep a history of selected Algorithms and datasets around in the toolchain
2022-03-04T09:39:05Z
Amir MOHAMMADI
Keep a history of selected Algorithms and datasets around in the toolchain
Summary
With the new editor, it's only possible to add blocks based on available datasets, algorithms, and analyzers. Which might be a good thing according to #265.
Also, once blocks are added (according to the algorithm or dataset it w...
Summary
With the new editor, it's only possible to add blocks based on available datasets, algorithms, and analyzers. Which might be a good thing according to #265.
Also, once blocks are added (according to the algorithm or dataset it was selected from), it is no longer associated with the block that it was chosen from.
This makes sense because as far as the toolchain is concerned, this association is irrelevant and this association is kept around in the experiment definition.
However, I think it makes sense to keep association around in the toolchain. Doing so can let us:
- Provide hints to the user (reminding them what algorithm that block was) so that they have to memorize less when designing the toolchain.
- Automatically figure out the channel of blocks as disccussed in #272.
On the other hand, we want to merge the toolchain/experiment definitions per https://gitlab.idiap.ch/beat/beat.core/-/issues/88
So maybe this issue must be looked at when that works is already done?!
https://gitlab.idiap.ch/beat/beat.editor/-/issues/273
Right-click actions on the blocks in the toolchain editor
2022-03-04T09:38:51Z
Amir MOHAMMADI
Right-click actions on the blocks in the toolchain editor
Here is my suggestion:
* Upon right clicking on the blocks, provide a list of actions.
* A `Delete` action
* A `Edit` or `Properties` action that does the same thing as double clicking
Here is my suggestion:
* Upon right clicking on the blocks, provide a list of actions.
* A `Delete` action
* A `Edit` or `Properties` action that does the same thing as double clicking
https://gitlab.idiap.ch/beat/beat.editor/-/issues/272
Automatic inference of synchronization channels when drawing the toolchain
2022-03-04T09:38:31Z
Amir MOHAMMADI
Automatic inference of synchronization channels when drawing the toolchain
Summary
When drawing several blocks in the toolchain and connecting inputs and outputs, the (synchronization) channel for blocks are not automatically determined even though most of the time there is only one option.
See the video below...
Summary
When drawing several blocks in the toolchain and connecting inputs and outputs, the (synchronization) channel for blocks are not automatically determined even though most of the time there is only one option.
See the video below where I have to select the channel in all blocks before getting a valid toolchain:
![synchronization_channels_not_selected_automatically](/uploads/75d3cce295e08c2515f69ace8c2810b8/synchronization_channels_not_selected_automatically.webm)
It would be beneficial if the toolchain editor would infer and select the channel for me as I edit along.
Why is it needed ?
It would make editing toolchains simpler as I will not have to go into each block and edit them one by one.
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/35
BEAT componenets are tied to a prefix
2022-03-03T17:35:06Z
Amir MOHAMMADI
BEAT componenets are tied to a prefix
Throughout all code and components of BEAT, a prefix is required and this requirement makes it impossible to define and run BEAT experiments interactively.
Here is a tentative plan for refactoring the code:
1. [ ] Update BEAT compone...
Throughout all code and components of BEAT, a prefix is required and this requirement makes it impossible to define and run BEAT experiments interactively.
Here is a tentative plan for refactoring the code:
1. [ ] Update BEAT component classes so that they can be created on the fly without pointing to a prefix
2. [ ] Implement a global config object to keep track of user's config such as where the prefix is or what the username is. This will help users provide less information when creating objects on the fly.
3. [ ] Dynamic creation of experiment/toolchain with running the python code in a kind of graph mode. This will be similar to how graphs are constructed in Python using tensorflow or dask.
3. [ ] We would also need a singleton class to hold the prefix objects in memory to avoid passing around caches.
https://gitlab.idiap.ch/beat/docs/-/issues/17
`beat editor serve` does not exist anymore
2021-04-08T12:14:33Z
André Anjos
`beat editor serve` does not exist anymore
There is a small typo on the front installation page referenced from other guides. It states `beat editor serve` should be available. Given the change in the editor strategy, I think it should read `beat editor start`.
There is a small typo on the front installation page referenced from other guides. It states `beat editor serve` should be available. Given the change in the editor strategy, I think it should read `beat editor start`.
https://gitlab.idiap.ch/beat/beat.core/-/issues/64
Database schema and runtime improvements
2021-01-29T09:46:59Z
André Anjos
Database schema and runtime improvements
Databases were originally designed monolithically, assuming the environment where packages necessary to read their contents is fixed (and does not change), and the location of files to be used for the readout is fixed. As an after thoug...
Databases were originally designed monolithically, assuming the environment where packages necessary to read their contents is fixed (and does not change), and the location of files to be used for the readout is fixed. As an after thought, templates were introduced without a proper formalisation and, as a consequence, templates with the same names may appear in various database JSON declarations, w/o necessarily being the same. This issue supersedes #25 and gathers all changes required for a revamp in this area:
- [x] Protocol and set templates should be separated from the database view declaration to avoid repetition and centralise "task-related" declarations. This will effectively simplify the declaration of new datasets
- [ ] The `root_db` parameter (maybe misspelled here) needs to be externalised as a runtime parameter during the run, as it is currently the case for other runtime prefixes such as algorithm caches. Currently, if one downloads the database view, they need to change this parameter by hand
- [x] The environment required to run the database view to provide the data for the experiment needs to be configured and have an entry on the database JSON declaration. This ensures that changes in the environment (docker image), will imply in new caches being generated at all times. As of today, it can happen that hashes are not regenerated even if the environment changes completely.
- [x] A default db env docker image (possibly named `beat.env.databases`) should be provided on docker hub. This avoids conflicts in future because when using multiple databases in one experiment, only one image can be used.
- [ ] The prototypes in `beat.core` should come configured to use this image.
- [ ] This new JSON entry must be documented in `beat/docs` and users should be recommended to use this image.
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.editor/-/issues/279
Search editor
2020-10-20T08:18:16Z
Samuel GAIST
Search editor
This issue tracks the implementation of the search feature of the platform inside beat/beat.editor>.
The goal of this feature is to allow people to directly manage searches from the editor and manipulate them there.
See beat/beat.web%...
This issue tracks the implementation of the search feature of the platform inside beat/beat.editor>.
The goal of this feature is to allow people to directly manage searches from the editor and manipulate them there.
See beat/beat.web%"Read-only platform"
Online feature editors
https://gitlab.idiap.ch/beat/beat.editor/-/issues/280
Attestation editor
2020-10-20T08:18:08Z
Samuel GAIST
Attestation editor
This issue tracks the implementation of the attestation feature of the platform inside beat/beat.editor>.
The goal of this feature is to allow people to directly manage attestations from the editor.
See beat/beat.web%"Read-only platform"
This issue tracks the implementation of the attestation feature of the platform inside beat/beat.editor>.
The goal of this feature is to allow people to directly manage attestations from the editor.
See beat/beat.web%"Read-only platform"
Online feature editors
https://gitlab.idiap.ch/beat/beat.editor/-/issues/281
Reports editor
2020-10-20T08:18:02Z
Samuel GAIST
Reports editor
This issue tracks the implementation of the reports feature of the platform inside beat/beat.editor>.
The goal of this feature is to allow people to create reports from the editor as well as manage the different related actions from the...
This issue tracks the implementation of the reports feature of the platform inside beat/beat.editor>.
The goal of this feature is to allow people to create reports from the editor as well as manage the different related actions from there.
See beat/beat.web%"Read-only platform"
Online feature editors
https://gitlab.idiap.ch/beat/beat.core/-/issues/74
Update the schemas to better reflect parameter type support in different objects
2020-10-19T15:16:45Z
Jaden DIEFENBAUGH
Update the schemas to better reflect parameter type support in different objects
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algor...
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algorithm schema, `algorithm/common.json:80`:
```json
80 "parameter": {
81 "type": "object",
82 "properties": {
83 "type": { "$ref": "../common/1.json#/definitions/basetype" },
84 "default": { "$ref": "../common/1.json#/definitions/value" },
85 "description": { "type": "string" }
86 },
87 "required": [
88 "type"
89 ]
90 },
```
The `type` field references the `basetype` types, which includes complex types, but the editors (`beat.web` & `beat.editor`) don't actually support complex types for algorithm parameters.
I'll be looking through the other schemas and will update this issue with other things I find, if any.
EDIT: didn't find anything else in the other objects' schemas parameter definitions
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.web/-/issues/567
Dead code removal
2020-09-28T09:20:24Z
Samuel GAIST
Dead code removal
Some code in this repository is no longer used.
For example scripts/scheduler.py. It has been replaced with Django management commands to take advantage of the ZMQ communication system.
Therefore, everything that is unused should be re...
Some code in this repository is no longer used.
For example scripts/scheduler.py. It has been replaced with Django management commands to take advantage of the ZMQ communication system.
Therefore, everything that is unused should be removed.
Code cleanup
https://gitlab.idiap.ch/beat/beat.web/-/issues/563
Code deduplication
2020-09-09T09:41:33Z
Samuel GAIST
Code deduplication
There are several places in beat/beat.web> where there is code duplication.
For example all interactive management commands follow the same structure and re-implement the `get_input_data`. This is typically a case where there should be ...
There are several places in beat/beat.web> where there is code duplication.
For example all interactive management commands follow the same structure and re-implement the `get_input_data`. This is typically a case where there should be a base class for that.
This would have avoided a mix of raw_input and input usage because of Python 2 VS Python 3 implementations.
Code cleanup
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/75
docker experiment runs not tested properly.
2020-08-10T15:27:01Z
Amir MOHAMMADI
docker experiment runs not tested properly.
There are several issues when you run the experiments with the docker executor:
* [x] `db.examples` docker image is not publicly available
* [ ] indexing of databases (`view.index` method) runs outside docker (different `root_folder` ...
There are several issues when you run the experiments with the docker executor:
* [x] `db.examples` docker image is not publicly available
* [ ] indexing of databases (`view.index` method) runs outside docker (different `root_folder` compared to inside docker containers) and this leads to broken paths when `.get` of views is called inside docker.
* [ ] logging leads to stat errors:
```
% beat experiment run --docker jmcgrat3/jmcgrat3/livdet-baseline/1/livdet-baseline
Index for database livdet-iris-2020/1 not found, building it
Registered 'LivDet2020 (1.0.0)' -> 'livdet2020:latest'
Untagged image, skipping
Registered 'Python (2.1.0)' -> 'beatenv/beat.env.python:2.1.0r7'
Registered 'Python (1.0.0)' -> 'beatenv/beat.env.python:latest'
Registered 'OpenCV (1.0.0)' -> 'opencv:latest'
Registered 'C++ (2.0.0)' -> 'beatenv/beat.env.cxx:2.0.0r3'
Running `jmcgrat3/BSIF-Feature-Extraction/1' for block `s17b12'
Start the execution of 'jmcgrat3/BSIF-Feature-Extraction/1'
Traceback (most recent call last):
File "/Users/josephmcgrath/miniconda3/envs/beat/bin/beat", line 11, in <module>
sys.exit(main())
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/beat/cmdline/decorators.py", line 94, in _decorator
value = view_func(*args, **kwargs)
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/beat/cmdline/experiments.py", line 782, in run
return run_experiment(config, name, force, docker, local, environment, quiet)
File "/Users/josephmcgrath/miniconda3/envs/beat/lib/python3.7/site-packages/beat/cmdline/experiments.py", line 312, in run_experiment
simplify_time(data_stats["time"]["read"]),
KeyError: 'time'
```
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/72
Local support for splittable
2020-08-06T14:39:48Z
Samuel GAIST
Local support for splittable
Currently there's no possibility to validate a splittable algorithm locally.
There's nothing in the experiment execution part and in the algorithm execution the number of slots is locked to one and not even used.
The splitting part is ...
Currently there's no possibility to validate a splittable algorithm locally.
There's nothing in the experiment execution part and in the algorithm execution the number of slots is locked to one and not even used.
The splitting part is currently done at the beat/beat.web> level and then the scheduler takes the generated tasks and sends them to the appropriate worker machines.
That part shall be ported to beat/beat.cmdline> so that the behaviour can be mimicked locally through maybe multiprocessing.
https://gitlab.idiap.ch/beat/beat.core/-/issues/100
Follow-up from "[livdet-iris-2020] Fix category type in view"
2020-08-05T08:35:55Z
Amir MOHAMMADI
Follow-up from "[livdet-iris-2020] Fix category type in view"
The following discussion from beat.examples!25 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/beat/beat.examples/-/merge_requests/25#note_53755): (+3 comments)
> btw the error for this was li...
The following discussion from beat.examples!25 should be addressed:
- [ ] @amohammadi started a [discussion](https://gitlab.idiap.ch/beat/beat.examples/-/merge_requests/25#note_53755): (+3 comments)
> btw the error for this was like this:
> ```
> $ beat exp run amohammadi/amohammadi/livdet/1/livedet
> Skipping execution of `amohammadi/livdet_baseline/1' for block `algorithm' - outputs exist
> Running `amohammadi/livdet_analyzer/1' for block `analyzer'
> Start the execution of 'amohammadi/livdet_analyzer/1'
> Block did not execute properly - outputs were reset
> Standard output:
>
> Standard error:
>
> Captured user error:
> File "/home/amir/miniconda/envs/iris/lib/python3.7/site-packages/beat/backend/python/baseformat.py", line 115, in setup_scalar
> % (attrname, formatname, numpy.array(value).dtype, dtype)
> TypeError: cannot safely cast attribute `value' on dataformat `system/integer/1' with type `int64' to `int32' without precision loss
> Captured system error:
>
> Error: Error occured: returned value is 1
> Removing cache files: No data written
> ```
> which was very hard to debug.
> The problem here is that the backtrace error I was getting was not telling me this error was in the view and not in my analyzer (the step I was on when hitting this error). So, I kept looking at my analyzer and thinking why I get this error while the error was in the view.
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/36
Editing the json file of database views does not invalidate the cache.
2020-07-28T09:45:09Z
Amir MOHAMMADI
Editing the json file of database views does not invalidate the cache.
Steps to reproduce:
```
$ beat cache clear
$ beat exp pull amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
$ beat exp run amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
Index for database atnt/5 not found, building it
li...
Steps to reproduce:
```
$ beat cache clear
$ beat exp pull amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
$ beat exp run amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
Index for database atnt/5 not found, building it
lib/python3.7/site-packages/setuptools/distutils_patch.py:26: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
"Distutils was imported before Setuptools. This usage is discouraged "
Index for database atnt/5 not found, building it
Index for database atnt/5 not found, building it
Running `amohammadi/pca/1' for block `linear_machine_training'
Start the execution of 'amohammadi/pca/1'
Block did not execute properly - outputs were reset
Standard output:
Standard error:
Captured user error:
File "lib/python3.7/site-packages/bob/io/base/__init__.py", line 143, in load
return File(inputs, 'r').read()
RuntimeError: File - constructor: C++ exception caught: 'file '/path_to_db_folder/att_faces/s1/1.pgm' is not readable'
Captured system error:
Error: Error occured: returned value is 1
Removing cache files: No data written
```
* change the json file `prefix/databases/atnt/5.json` of the database and fix the the `root_folder`
* run the experiment again:
```
$ beat exp run amohammadi/tutorial/eigenface/1/atnt-eigenfaces-67-comp
RuntimeError: File - constructor: C++ exception caught: 'file '/path_to_db_folder/att_faces/s1/1.pgm' is not readable'
```
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/74
Pushin an existing experiment leads to a 'description' KeyError
2020-07-22T13:23:47Z
Amir MOHAMMADI
Pushin an existing experiment leads to a 'description' KeyError
```
pushing experiments/amohammadi/amohammadi/livdet/1/livdet [ModificationStatus.DOC_CHANGED]
Traceback (most recent call last):
File "bin/beat", line 11, in <module>
sys.exit(main())
File "lib/python3.7/site-packages/click/core...
```
pushing experiments/amohammadi/amohammadi/livdet/1/livdet [ModificationStatus.DOC_CHANGED]
Traceback (most recent call last):
File "bin/beat", line 11, in <module>
sys.exit(main())
File "lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "lib/python3.7/site-packages/beat/cmdline/click_helper.py", line 121, in invoke
return super().invoke(ctx)
File "lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "lib/python3.7/site-packages/beat/cmdline/decorators.py", line 94, in _decorator
value = view_func(*args, **kwargs)
File "lib/python3.7/site-packages/beat/cmdline/commands.py", line 321, in push_impl
indentation=0,
File "lib/python3.7/site-packages/beat/cmdline/common.py", line 1497, in push
data = {"description": message["description"]}
KeyError: 'description'
```
I am not sure what is the desired behavior here because the experiment already exists and is run.
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.examples/-/issues/16
Create a new version of all databases and set the database environment they s...
2020-07-07T12:08:20Z
Samuel GAIST
Create a new version of all databases and set the database environment they should use
With the latest version of beat/beat.core> databases can now list which environment to use.
This will allow to avoid having to make an update to the corresponding images to list each time a new database created.
For backward compatibil...
With the latest version of beat/beat.core> databases can now list which environment to use.
This will allow to avoid having to make an update to the corresponding images to list each time a new database created.
For backward compatibility, will keep the old declaration as is as the cash was built with them. The new version implies that the index must be regenerated.