beat issues
https://gitlab.idiap.ch/groups/beat/-/issues
2020-06-08T14:00:56Z
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/33
Add multi-processing support to RemoteDataSource
2020-06-08T14:00:56Z
Samuel GAIST
Add multi-processing support to RemoteDataSource
Following #32
The current implementation of the processing block to database container uses a one to one connection through ZMQ exclusive pair.
In order to also allow people to use multiprocessing with a dataset as input to a block, th...
Following #32
The current implementation of the processing block to database container uses a one to one connection through ZMQ exclusive pair.
In order to also allow people to use multiprocessing with a dataset as input to a block, this much be changed to something different. Essentially multiple clients to one server in request/response mode.
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/69
Cannot create a new database
2020-06-02T14:14:34Z
Amir MOHAMMADI
Cannot create a new database
Running `beat databases create myiris/1` will result in:
```
Traceback (most recent call last):
File "/home/amir/miniconda/envs/beat-tutorial/bin/beat", line 11, in <module>
sys.exit(main())
File "/home/amir/miniconda/envs/beat-t...
Running `beat databases create myiris/1` will result in:
```
Traceback (most recent call last):
File "/home/amir/miniconda/envs/beat-tutorial/bin/beat", line 11, in <module>
sys.exit(main())
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/cmdline/click_helper.py", line 121, in invoke
return super().invoke(ctx)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/cmdline/decorators.py", line 92, in _decorator
value = view_func(*args, **kwargs)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/cmdline/commands.py", line 162, in create_impl
return common.create(config.path, asset_info.asset_type, names)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/cmdline/common.py", line 1009, in create
obj = TYPE_VALIDATOR[asset_type](prefix, data=None) # the default object
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/core/database.py", line 131, in __init__
super(Database, self).__init__(prefix, data, dataformat_cache)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/backend/python/database.py", line 264, in __init__
self._load(name, dataformat_cache)
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/core/database.py", line 167, in _load
self.prefix
File "/home/amir/miniconda/envs/beat-tutorial/lib/python3.7/site-packages/beat/core/database.py", line 69, in get_first_procotol_template
raise RuntimeError("Invalid prefix content, no protocol template available")
RuntimeError: Invalid prefix content, no protocol template available
```
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/68
beat exp fork command is cumbersome to use
2020-05-15T12:56:46Z
Amir MOHAMMADI
beat exp fork command is cumbersome to use
When you fork an experiment in the command line, you have to something like:
```sh
$ beat exp fork test/test/iris/1/iris amir/test/iris/1/means
```
but you can only change the username and name of the experiment when you fork.
When usin...
When you fork an experiment in the command line, you have to something like:
```sh
$ beat exp fork test/test/iris/1/iris amir/test/iris/1/means
```
but you can only change the username and name of the experiment when you fork.
When using beat.editor 2.0.0, when I fork an experiment, it only asks for the exp name.
Beat.editor somehow decided to put the forked experiment under user `amir` but I don't know
yet how it decided on that.
Shouldn't this command be also similar?
Something like:
```sh
$ beat exp fork test/test/iris/1/iris means
$ beat exp fork --user amir test/test/iris/1/iris means
```
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/31
Support bytes python type
2020-05-27T16:23:17Z
Samuel GAIST
Support bytes python type
Currently there's no way to send bytes directly from one algorithm to another.
The current "workaround" is to build the bytes output then use base64 encoding, use a string based type to write the data to the output and on the other end ...
Currently there's no way to send bytes directly from one algorithm to another.
The current "workaround" is to build the bytes output then use base64 encoding, use a string based type to write the data to the output and on the other end load it back from string.
Example of working code:
Output:
```python
obj = dict(field1=data.value*2, field2=data.value)
dumped = pickle.dumps(obj) # binary
encoded = base64.b64encode(dumped) # base64 bytes
string = encoded.decode("ascii") # "stringified"
outputs["out_data"].write({"value": string})
```
**WARNING** Do **not** call str(encoded), it won't be a "real string":
```python
example = b"whatever"
example_str = str(example)
example_str
# output is "b'whatever'"
```
Input:
```python
in_data = inputs["in_data"].data # string data
decoded = base64.b64decode(in_data.value) # decode to bytes
obj = pickle.loads(decoded) # obj is a dict and can be read
```
This issue tracks the implementation of the support for the python bytes type.
https://gitlab.idiap.ch/beat/beat.core/-/issues/92
Simplifying toolchain connectivity
2019-11-18T09:14:34Z
André Anjos
Simplifying toolchain connectivity
Thought about dumping this here, for tracking.
While discussing with @flavio.tarsetti on the toolchain editor and @tiago.pereira on snakemake and workflow definitions, a thought occurred that may simplify our lives and those of our user...
Thought about dumping this here, for tracking.
While discussing with @flavio.tarsetti on the toolchain editor and @tiago.pereira on snakemake and workflow definitions, a thought occurred that may simplify our lives and those of our users a bit: why not avoiding this discrete port to port connection between blocks and accept "whole" block connections only?
The idea is to design toolchains only based on block connections instead of block input/outputs. As a result, a block that receives as an input data from another block, may access any of its outputs.
I'm not sure about the technical challenge(s) behind this, but that would definitely simplify the toolchain editor a bit.
My current understanding is that this does not affect block synchronization and may be relatively "easy" to implement.
@samuel.gaist: what do you think? Easy to roll out?
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.web/-/issues/536
Some of the Models defined in django are not well documented
2019-09-05T11:36:57Z
Zohreh MOSTAANI
Some of the Models defined in django are not well documented
The "swagger.yaml" file that includes the beat.web api information that can be rendered into the sphinx documentation does not include complete information for all the models included in a response object. Some of the basic ones are show...
The "swagger.yaml" file that includes the beat.web api information that can be rendered into the sphinx documentation does not include complete information for all the models included in a response object. Some of the basic ones are shown though which means that the package produces this yaml file is working as expected.
It seems that those objects which are not shown, are not well documented in the code and because there are several layers of inheritance for each class, it is difficult to find where exactly the problem lies.
With a brief discussion with @samuel.gaist we tried something fast but it did not work. It required a deeper dive into the implementation.
https://gitlab.idiap.ch/beat/beat.web/-/issues/533
Strange behaviour concerning a POST method that needed to be GET.
2019-08-22T15:38:08Z
Zohreh MOSTAANI
Strange behaviour concerning a POST method that needed to be GET.
In the `api.py` file for `report`, there are two classes that have POST methods which in fact should have been GET.
`ReportRSTCompileView` and `ReportRSTCompileAnonView`. The definitions are here:
https://gitlab.idiap.ch/beat/beat.web/b...
In the `api.py` file for `report`, there are two classes that have POST methods which in fact should have been GET.
`ReportRSTCompileView` and `ReportRSTCompileAnonView`. The definitions are here:
https://gitlab.idiap.ch/beat/beat.web/blob/master/beat/web/reports/api.py
Both of the mentioned classes are using another method `compileTextItem` defined in `models.py`. This function is not readable and it raises an exception error `"Error in ReportRSTCompileView: The Python docutils library isn't installed."` which does not seem to belong here. The definition is here:
https://gitlab.idiap.ch/beat/beat.web/blob/master/beat/web/reports/models.py
https://gitlab.idiap.ch/beat/beat.editor/-/issues/228
Feature request: Be able to have multi window open
2020-02-12T12:42:39Z
Zohreh MOSTAANI
Feature request: Be able to have multi window open
It would be nice if you are able to have multiple assets open at the same time, since sometimes you may need it.
There are to options here:
1. Having multiple windows open in the same instance of editor. Kind of just splitting the edit...
It would be nice if you are able to have multiple assets open at the same time, since sometimes you may need it.
There are to options here:
1. Having multiple windows open in the same instance of editor. Kind of just splitting the editor window.
2. Having a new instance of editor opened from terminal. In this case there might be complications with the log info shown in the terminal. The question is are they useful for user or not?
https://gitlab.idiap.ch/beat/beat.examples/-/issues/13
Update all algorithms to be V2
2019-05-20T07:17:55Z
Samuel GAIST
Update all algorithms to be V2
Currently all the algorithms examples are V1. They need to be updated to the V2 API.
The straightforward way to do that is to make them all sequential algorithms.
Currently all the algorithms examples are V1. They need to be updated to the V2 API.
The straightforward way to do that is to make them all sequential algorithms.
https://gitlab.idiap.ch/beat/beat.editor/-/issues/217
Update editor needing complex type support
2020-02-12T13:15:12Z
Samuel GAIST
Update editor needing complex type support
The latest version of beat/beat.core> following beat/beat.core!65 doesn't contain complex types anymore as they are specific to the Dataformat asset.
The DataformatEditor must be updated to handle that change.
The latest version of beat/beat.core> following beat/beat.core!65 doesn't contain complex types anymore as they are specific to the Dataformat asset.
The DataformatEditor must be updated to handle that change.
[v2] 1 - Edition/Visualization for small editors
https://gitlab.idiap.ch/beat/beat.editor/-/issues/216
Update basetype getter
2020-02-12T13:14:31Z
Samuel GAIST
Update basetype getter
beat/beat.core!65 modifies heavily the schema to be more accurate.
The basetype definition changes in that regards and the current setter doesn't support it yet.
beat/beat.core!65 modifies heavily the schema to be more accurate.
The basetype definition changes in that regards and the current setter doesn't support it yet.
[v2] 1 - Edition/Visualization for small editors
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/60
Add support for protocol template
2019-05-08T09:45:49Z
Samuel GAIST
Add support for protocol template
For database v2, the new protocol template object has been created (see beat/beat.core#64, beat/beat.core!67, and beat/beat.core!71).
This issue tracks adding support for it in beat/beat.cmdline>
Todo:
- [x] Local support
- [ ] Remote ...
For database v2, the new protocol template object has been created (see beat/beat.core#64, beat/beat.core!67, and beat/beat.core!71).
This issue tracks adding support for it in beat/beat.cmdline>
Todo:
- [x] Local support
- [ ] Remote support
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/76
Clean up the schemas
2019-06-02T02:18:24Z
Jaden DIEFENBAUGH
Clean up the schemas
(@samuel.gaist this is the issue for our email discussion)
Both !55 and !65 showed that there were places to improve the schema. This issue expands on those MRs' work by using the v6/v7 json schema features (where appropriate) and addin...
(@samuel.gaist this is the issue for our email discussion)
Both !55 and !65 showed that there were places to improve the schema. This issue expands on those MRs' work by using the v6/v7 json schema features (where appropriate) and adding tests to complicated parts of the schema. Right now I know of a couple of things that need to be done:
- [x] Use the `const` schema keyword instead of the `enum` with only one choice. This issue is documented in the schema, e.g. in beat.core/beat/core/schema/algorithm/3.json:
```json
115- "type": {
116: "$comment": "Change enum to const when tools allow v6 json schema",
117- "type": "string",
118- "enum": ["loop_user"]
119- }
--
144- "type": {
145: "$comment": "Change enum to const when tools allow v6 json schema",
146- "type": "string",
147- "enum": ["loop"]
148- }
```
- [ ] Use the `if`/`then`/`else` keywords from json schema v7 to represent more complex logic (e.g. the changes to algorithm parameters in !65)
- As found in !65, there are parts of the schema that aren't tested thoroughly. This is nebulous issue, however, so @flavio.tarsetti @samuel.gaist could you maybe specify what I should do here, so I don't go off the rails?
(As other issues are isolated, I'll add them to the list)
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.core/-/issues/74
Update the schemas to better reflect parameter type support in different objects
2020-10-19T15:16:45Z
Jaden DIEFENBAUGH
Update the schemas to better reflect parameter type support in different objects
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algor...
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algorithm schema, `algorithm/common.json:80`:
```json
80 "parameter": {
81 "type": "object",
82 "properties": {
83 "type": { "$ref": "../common/1.json#/definitions/basetype" },
84 "default": { "$ref": "../common/1.json#/definitions/value" },
85 "description": { "type": "string" }
86 },
87 "required": [
88 "type"
89 ]
90 },
```
The `type` field references the `basetype` types, which includes complex types, but the editors (`beat.web` & `beat.editor`) don't actually support complex types for algorithm parameters.
I'll be looking through the other schemas and will update this issue with other things I find, if any.
EDIT: didn't find anything else in the other objects' schemas parameter definitions
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.core/-/issues/72
Assert usage cleanup
2019-05-09T13:26:49Z
Samuel GAIST
Assert usage cleanup
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.core.
Depending on how they are used, the following actions can be done:
1. Turn the statement into ...
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.core.
Depending on how they are used, the following actions can be done:
1. Turn the statement into an exception
2. Handle the case gracefully
3. Mark as not to get worried about
The current state of the code base is usually 3 which should be cleaned up.
All tests in this repository have been cleaned to use the testing framework corresponding tools
https://gitlab.idiap.ch/beat/beat.editor/-/issues/196
rearrange the inputs in a toolchain block
2020-06-16T16:03:32Z
Zohreh MOSTAANI
rearrange the inputs in a toolchain block
It would be very useful if the user is able to reorder the inputs when they are defined in a block. Sometimes you may just make a block with the inputs but it will be easier to connect them to other blocks if they were in another order. ...
It would be very useful if the user is able to reorder the inputs when they are defined in a block. Sometimes you may just make a block with the inputs but it will be easier to connect them to other blocks if they were in another order. This is different from renaming though.
[v2] 2 - Toolchain editor
https://gitlab.idiap.ch/beat/beat.editor/-/issues/193
breaking connections in a toolchain to by pass the blocks
2020-02-12T12:52:21Z
Zohreh MOSTAANI
breaking connections in a toolchain to by pass the blocks
I think it would be good if the connections in a toolchain could break at some points and bypass blocks instead of directly connecting inputs and outputs. Something like what is available on the beat platform. The reason is that even tho...
I think it would be good if the connections in a toolchain could break at some points and bypass blocks instead of directly connecting inputs and outputs. Something like what is available on the beat platform. The reason is that even though it is more convenient while making a toolchain to just connect an output to an input it is very difficult to understand what is going on when you look at an experiment using that toolchain, specially when the experiments get bigger and the connections are crossing each other. Even if there was a mechanism to highlight a chosen connection in the experiment page can be really helpful. This mechanism is already there when making a toolchain but not when you look at the experiment.
[v2] 2 - Toolchain editor
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/56
There is no way to make a clean prefix directory structure with only trusted ...
2018-12-21T14:52:06Z
Zohreh MOSTAANI
There is no way to make a clean prefix directory structure with only trusted system files.
In the ALLIES meeting one of the points that was raised was that how to get a clean prefix directory structure, with databases, dataformats, etc in place. With only data from the system not other users, and not to go through copying a pr...
In the ALLIES meeting one of the points that was raised was that how to get a clean prefix directory structure, with databases, dataformats, etc in place. With only data from the system not other users, and not to go through copying a prefix from another git repository.
It may be possible to use command line to make such a directory.
https://gitlab.idiap.ch/beat/docs/-/issues/10
Databases: How to pass parameters to the Index method.
2018-12-21T14:43:32Z
Zohreh MOSTAANI
Databases: How to pass parameters to the Index method.
There is not enough information about the parameters that can be passed to the index method and not enough examples.
There is not enough information about the parameters that can be passed to the index method and not enough examples.
https://gitlab.idiap.ch/beat/docs/-/issues/9
How to test the workflow in more details.
2018-12-21T14:45:38Z
Zohreh MOSTAANI
How to test the workflow in more details.
There is not enough information how to test the workflow. Also maybe how to check each block with minimum overhead.
There is not enough information how to test the workflow. Also maybe how to check each block with minimum overhead.