beat issues
https://gitlab.idiap.ch/groups/beat/-/issues
2018-05-01T09:09:50Z
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/23
[cache] experiments data does not contain all information such as "type" field
2018-05-01T09:09:50Z
Flavio TARSETTI
[cache] experiments data does not contain all information such as "type" field
When the experiments is run locally,
We have a major issue.
Why aren't the information saved the same way as in the API?
One of the most important information is missing in the cache.
I was expecting this as in the API:
"results...
When the experiments is run locally,
We have a major issue.
Why aren't the information saved the same way as in the API?
One of the most important information is missing in the cache.
I was expecting this as in the API:
"results": {
"analysis": {
"bar_plot": {
"type": "plot/bar/1",
"primary": false,
"value": {
"data": [
{
"y": [
0.0,
1.0,
2.0,
But I was able to get only this from the local cache:
"bar_plot": {
"data": [
{
"y": [
0.0,
1.0,
2.0,
3.0,
Where is the "type" field gone?
The type field is important as it indicates which plotter can handle this kind of data.
At present, I am working on plotting using data I get remotely from the experiment when hitting the API, but someone would want to be able to use experiments data he has ran locally and this is not possible at the moment due to the "cache" missing information. This is blocking for local plotting from locally ran experiments.
https://gitlab.idiap.ch/beat/beat.core/-/issues/56
Docker timer tests rely on having adequate CI resources
2018-12-12T09:13:59Z
Jaden DIEFENBAUGH
Docker timer tests rely on having adequate CI resources
Sometimes docker timing tests will fail when the tests go too slowly. These tests rely on a hard-coded number of seconds/milliseconds with numbers that are reasonable to assume on one's own machine but obviously aren't designed for a CI ...
Sometimes docker timing tests will fail when the tests go too slowly. These tests rely on a hard-coded number of seconds/milliseconds with numbers that are reasonable to assume on one's own machine but obviously aren't designed for a CI (just see the [docker tests](https://gitlab.idiap.ch/beat/beat.core/blob/85cef289db0eac7aaa47308dd92cc5003a6a689b/beat/core/test/test_docker.py).
Some of these tests can just be changed. For example, the `test_images_cache` tests are actually relative tests - you just want to be sure that doing something a second time is faster than the first time. I believe that the CI performance would be consistent across each of these tests (which shouldn't last for more than a couple seconds).
There's other ones though, like the timeout tests. Not sure what to do with those.
This doesn't unconditionally make the CI fail since rerunning the job usually fixes it.
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/27
[all packages] code cleanup
2018-09-04T09:09:56Z
Flavio TARSETTI
[all packages] code cleanup
beat.cmdline needs to get some things changed such as:
- type keyword to remove
- status = None returned (to return a real status)
beat.cmdline needs to get some things changed such as:
- type keyword to remove
- status = None returned (to return a real status)
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/35
Add sort option for `beat <subcommand> list`
2018-05-30T08:24:38Z
Jaden DIEFENBAUGH
Add sort option for `beat <subcommand> list`
Sort by:
- Name
- Last edited
- ???
Sort by:
- Name
- Last edited
- ???
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/36
Add `--long` flag when listing things
2018-05-30T08:28:19Z
Jaden DIEFENBAUGH
Add `--long` flag when listing things
It's just like `ls`'s `-l` flag:
- last edited
- maybe which filetypes the object has associated with it?
It's just like `ls`'s `-l` flag:
- last edited
- maybe which filetypes the object has associated with it?
https://gitlab.idiap.ch/beat/beat.core/-/issues/57
Is `return True` needed in the algorithms API still?
2018-05-30T14:33:58Z
Jaden DIEFENBAUGH
Is `return True` needed in the algorithms API still?
Not sure where I should be putting this issue...
Right now, as far as I know, `process` methods in algorithms must return `True`. Is this still required?
@samuel.gaist
Not sure where I should be putting this issue...
Right now, as far as I know, `process` methods in algorithms must return `True`. Is this still required?
@samuel.gaist
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/37
`exp caches --list` doesn't print out db caches
2018-05-30T15:22:15Z
Jaden DIEFENBAUGH
`exp caches --list` doesn't print out db caches
When printing caches for an experiment, it prints out all blocks' caches except the dataset blocks' caches.
When printing caches for an experiment, it prints out all blocks' caches except the dataset blocks' caches.
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/39
Better copy/paste support for `exp caches`
2018-06-11T08:13:15Z
Jaden DIEFENBAUGH
Better copy/paste support for `exp caches`
Right now the caches printed are file paths:
```
block: `pre_training'
algorithm: `test/iris_preprocessor/1'
output: `/home/jae/code/beat.editor/prefix/cache/a9/e2/0b/2ba5928422351e52a508bd0cbc55e3d0e5bacb05896dc28413bcd0a1a4'
block:...
Right now the caches printed are file paths:
```
block: `pre_training'
algorithm: `test/iris_preprocessor/1'
output: `/home/jae/code/beat.editor/prefix/cache/a9/e2/0b/2ba5928422351e52a508bd0cbc55e3d0e5bacb05896dc28413bcd0a1a4'
block: `pre_testing'
algorithm: `test/iris_preprocessor/1'
output: `/home/jae/code/beat.editor/prefix/cache/1e/a5/74/80c7d48bae3d0f6baf0883c3a7f78b5324830931029fcee4fd3bffff4d'
block: `training_alg'
algorithm: `test/iris_training/1'
output: `/home/jae/code/beat.editor/prefix/cache/45/61/15/3438e272598bf15e6bd729709536f4ca38b2b9d1061dd286c6dae5f288'
block: `testing_alg'
algorithm: `test/iris_testing/1'
output: `/home/jae/code/beat.editor/prefix/cache/39/d4/20/a6709aca3cac7e9a9a7cb2c36d536184554b5e19358927f2e866ad5751'
block: `analyzer'
analyzer: `test/iris_analyzer/1'
output: `/home/jae/code/beat.editor/prefix/cache/f0/a0/86/a645856dcbee4edc2729fcc60554212ed5def33334670602be6086aafb'
```
But you only use path segments relative to the cache folder to operate on that cache:
```
$ beat ca view a9/e2/0b/2ba5928422351e52a508bd0cbc55e3d0e5bacb05896dc28413bcd0a1a4
```
It would make sense for `exp caches` to only print out this bit, not the absolute path.
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/53
strange behaviour of beat.cmline when pulling objects from platform results i...
2018-12-06T09:53:24Z
Zohreh MOSTAANI
strange behaviour of beat.cmline when pulling objects from platform results in beat.editor crashing as well.
Hi,
Today when I ran beateditor command to edit some experiment, it did not show me anything on the Firefox. It only shows the page for split second and then it is a blank page. I updated beat.editor with conda in case there were some c...
Hi,
Today when I ran beateditor command to edit some experiment, it did not show me anything on the Firefox. It only shows the page for split second and then it is a blank page. I updated beat.editor with conda in case there were some changes but the problem is still there.
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/46
plot function does not work properly if the prefix is set with relative path.
2018-09-06T14:46:40Z
Zohreh MOSTAANI
plot function does not work properly if the prefix is set with relative path.
Hi,
I set my prefix to a relative path by using `beat config set` command. I can run the experiment but When I try to plot the result I get this error, it looks in the wrong directory.
```
retrieving `plotters/plotterparameters/plot/is...
Hi,
I set my prefix to a relative path by using `beat config set` command. I can run the experiment but When I try to plot the result I get this error, it looks in the wrong directory.
```
retrieving `plotters/plotterparameters/plot/isoroc/1'...
Traceback (most recent call last):
File "/home/zmostaani/miniconda3/envs/beat_mini/bin/beat", line 11, in <module>
sys.exit(main())
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/decorators.py", line 75, in _decorator
value = view_func(*args, **kwargs)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/experiments.py", line 783, in plot
output_folder
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/experiments.py", line 535, in plot_impl
data.as_dict()[the_data], output_name, None, indentation + 2, format_cache)
File "/home/zmostaani/miniconda3/envs/beat_mini/lib/python3.6/site-packages/beat/cmdline/plotters.py", line 238, in plot_impl
with open(output_name, "wb") as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'beat.zmostaani.prefix/prefix/beat.zmostaani.prefix/prefix/experiments/zmostaani/tutorial/eigenface/1/atnt-eigenfaces-75-comp-bis-vbox_roc.png'
```
in this example the my prefix value is `"prefix": "beat.zmostaani.prefix/prefix"` and I am running the experiment on the parent folder of beat.zmostaani.prefix.
I tried with different relative paths and I get the similar error, however if I use the absolute path everything is ok.
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/51
The connections are not pushed to the platform when pushing a toolchain.
2018-12-04T15:35:40Z
Zohreh MOSTAANI
The connections are not pushed to the platform when pushing a toolchain.
I pushed a toolchain to the platform with this command:
```
beat toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
```
This is the output:
```
toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
[+] toolchains/tut...
I pushed a toolchain to the platform with this command:
```
beat toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
```
This is the output:
```
toolchains push zmostaani/sbhatta-iqm-face-antispoofing-test/2
[+] toolchains/tutorial/eigenface/1 (@2014-08-11T14:27:37)
[l] toolchains/test/iris_advanced_editor/1
[l] toolchains/zmostaani/eigenface-new-error/2
[l] toolchains/tutorial/eigenface-error/1
[l] toolchains/zmostaani/editor-test/1
[l] toolchains/zmostaani/editor-test-2/1
[l] toolchains/zmostaani/eigenface-clone/1
[l] toolchains/zmostaani/eigenface-test/1
[l] toolchains/test/test/1
[l] toolchains/zmostaani/eigenface-new/1
[l] toolchains/zmostaani/sbhatta-iqm-face-antispoofing-test/1
[l] toolchains/test/iris/1
[l] toolchains/zmostaani/sbhatta-iqm-face-antispoofing-test/2
[l] toolchains/test/iris_advanced/1
[l] toolchains/zmostaani/eigenface-new-error/1
pushing toolchains/zmostaani/sbhatta-iqm-face-antispoofing-test/2 [l]
Successfuly created `zmostaani/sbhatta-iqm-face-antispoofing-test/2'
```
I went to check it on the platform. Aside from the problem with version numbering that has been mentioned in issue #49 and #50, when I look at it on the platform there is no connection to any block.
https://gitlab.idiap.ch/beat/beat.core/-/issues/64
Database schema and runtime improvements
2021-01-29T09:46:59Z
André Anjos
Database schema and runtime improvements
Databases were originally designed monolithically, assuming the environment where packages necessary to read their contents is fixed (and does not change), and the location of files to be used for the readout is fixed. As an after thoug...
Databases were originally designed monolithically, assuming the environment where packages necessary to read their contents is fixed (and does not change), and the location of files to be used for the readout is fixed. As an after thought, templates were introduced without a proper formalisation and, as a consequence, templates with the same names may appear in various database JSON declarations, w/o necessarily being the same. This issue supersedes #25 and gathers all changes required for a revamp in this area:
- [x] Protocol and set templates should be separated from the database view declaration to avoid repetition and centralise "task-related" declarations. This will effectively simplify the declaration of new datasets
- [ ] The `root_db` parameter (maybe misspelled here) needs to be externalised as a runtime parameter during the run, as it is currently the case for other runtime prefixes such as algorithm caches. Currently, if one downloads the database view, they need to change this parameter by hand
- [x] The environment required to run the database view to provide the data for the experiment needs to be configured and have an entry on the database JSON declaration. This ensures that changes in the environment (docker image), will imply in new caches being generated at all times. As of today, it can happen that hashes are not regenerated even if the environment changes completely.
- [x] A default db env docker image (possibly named `beat.env.databases`) should be provided on docker hub. This avoids conflicts in future because when using multiple databases in one experiment, only one image can be used.
- [ ] The prototypes in `beat.core` should come configured to use this image.
- [ ] This new JSON entry must be documented in `beat/docs` and users should be recommended to use this image.
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/docs/-/issues/9
How to test the workflow in more details.
2018-12-21T14:45:38Z
Zohreh MOSTAANI
How to test the workflow in more details.
There is not enough information how to test the workflow. Also maybe how to check each block with minimum overhead.
There is not enough information how to test the workflow. Also maybe how to check each block with minimum overhead.
https://gitlab.idiap.ch/beat/docs/-/issues/10
Databases: How to pass parameters to the Index method.
2018-12-21T14:43:32Z
Zohreh MOSTAANI
Databases: How to pass parameters to the Index method.
There is not enough information about the parameters that can be passed to the index method and not enough examples.
There is not enough information about the parameters that can be passed to the index method and not enough examples.
https://gitlab.idiap.ch/beat/beat.cmdline/-/issues/56
There is no way to make a clean prefix directory structure with only trusted ...
2018-12-21T14:52:06Z
Zohreh MOSTAANI
There is no way to make a clean prefix directory structure with only trusted system files.
In the ALLIES meeting one of the points that was raised was that how to get a clean prefix directory structure, with databases, dataformats, etc in place. With only data from the system not other users, and not to go through copying a pr...
In the ALLIES meeting one of the points that was raised was that how to get a clean prefix directory structure, with databases, dataformats, etc in place. With only data from the system not other users, and not to go through copying a prefix from another git repository.
It may be possible to use command line to make such a directory.
https://gitlab.idiap.ch/beat/beat.editor/-/issues/193
breaking connections in a toolchain to by pass the blocks
2020-02-12T12:52:21Z
Zohreh MOSTAANI
breaking connections in a toolchain to by pass the blocks
I think it would be good if the connections in a toolchain could break at some points and bypass blocks instead of directly connecting inputs and outputs. Something like what is available on the beat platform. The reason is that even tho...
I think it would be good if the connections in a toolchain could break at some points and bypass blocks instead of directly connecting inputs and outputs. Something like what is available on the beat platform. The reason is that even though it is more convenient while making a toolchain to just connect an output to an input it is very difficult to understand what is going on when you look at an experiment using that toolchain, specially when the experiments get bigger and the connections are crossing each other. Even if there was a mechanism to highlight a chosen connection in the experiment page can be really helpful. This mechanism is already there when making a toolchain but not when you look at the experiment.
[v2] 2 - Toolchain editor
https://gitlab.idiap.ch/beat/beat.editor/-/issues/196
rearrange the inputs in a toolchain block
2020-06-16T16:03:32Z
Zohreh MOSTAANI
rearrange the inputs in a toolchain block
It would be very useful if the user is able to reorder the inputs when they are defined in a block. Sometimes you may just make a block with the inputs but it will be easier to connect them to other blocks if they were in another order. ...
It would be very useful if the user is able to reorder the inputs when they are defined in a block. Sometimes you may just make a block with the inputs but it will be easier to connect them to other blocks if they were in another order. This is different from renaming though.
[v2] 2 - Toolchain editor
https://gitlab.idiap.ch/beat/beat.core/-/issues/72
Assert usage cleanup
2019-05-09T13:26:49Z
Samuel GAIST
Assert usage cleanup
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.core.
Depending on how they are used, the following actions can be done:
1. Turn the statement into ...
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.core.
Depending on how they are used, the following actions can be done:
1. Turn the statement into an exception
2. Handle the case gracefully
3. Mark as not to get worried about
The current state of the code base is usually 3 which should be cleaned up.
All tests in this repository have been cleaned to use the testing framework corresponding tools
https://gitlab.idiap.ch/beat/beat.core/-/issues/74
Update the schemas to better reflect parameter type support in different objects
2020-10-19T15:16:45Z
Jaden DIEFENBAUGH
Update the schemas to better reflect parameter type support in different objects
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algor...
See https://gitlab.idiap.ch/beat/beat.editor/issues/205#note_40301
The issue we found first is that the algorithm editor's parameter logic doesn't actually support complex types, but the algorithm schema indicates support. See the algorithm schema, `algorithm/common.json:80`:
```json
80 "parameter": {
81 "type": "object",
82 "properties": {
83 "type": { "$ref": "../common/1.json#/definitions/basetype" },
84 "default": { "$ref": "../common/1.json#/definitions/value" },
85 "description": { "type": "string" }
86 },
87 "required": [
88 "type"
89 ]
90 },
```
The `type` field references the `basetype` types, which includes complex types, but the editors (`beat.web` & `beat.editor`) don't actually support complex types for algorithm parameters.
I'll be looking through the other schemas and will update this issue with other things I find, if any.
EDIT: didn't find anything else in the other objects' schemas parameter definitions
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH
https://gitlab.idiap.ch/beat/beat.core/-/issues/76
Clean up the schemas
2019-06-02T02:18:24Z
Jaden DIEFENBAUGH
Clean up the schemas
(@samuel.gaist this is the issue for our email discussion)
Both !55 and !65 showed that there were places to improve the schema. This issue expands on those MRs' work by using the v6/v7 json schema features (where appropriate) and addin...
(@samuel.gaist this is the issue for our email discussion)
Both !55 and !65 showed that there were places to improve the schema. This issue expands on those MRs' work by using the v6/v7 json schema features (where appropriate) and adding tests to complicated parts of the schema. Right now I know of a couple of things that need to be done:
- [x] Use the `const` schema keyword instead of the `enum` with only one choice. This issue is documented in the schema, e.g. in beat.core/beat/core/schema/algorithm/3.json:
```json
115- "type": {
116: "$comment": "Change enum to const when tools allow v6 json schema",
117- "type": "string",
118- "enum": ["loop_user"]
119- }
--
144- "type": {
145: "$comment": "Change enum to const when tools allow v6 json schema",
146- "type": "string",
147- "enum": ["loop"]
148- }
```
- [ ] Use the `if`/`then`/`else` keywords from json schema v7 to represent more complex logic (e.g. the changes to algorithm parameters in !65)
- As found in !65, there are parts of the schema that aren't tested thoroughly. This is nebulous issue, however, so @flavio.tarsetti @samuel.gaist could you maybe specify what I should do here, so I don't go off the rails?
(As other issues are isolated, I'll add them to the list)
Jaden DIEFENBAUGH
Jaden DIEFENBAUGH