beat issues
https://gitlab.idiap.ch/groups/beat/-/issues
2020-07-29T09:18:53Z
https://gitlab.idiap.ch/beat/beat.core/-/issues/101
Add tmpfs when running a container
2020-07-29T09:18:53Z
Samuel GAIST
Add tmpfs when running a container
Despite the tests run in the various aspects of the BEAT code base which showed that having the container start read-only was successful, actual run on the platform started to unsuccessfully fail with an error related to temporary file c...
Despite the tests run in the various aspects of the BEAT code base which showed that having the container start read-only was successful, actual run on the platform started to unsuccessfully fail with an error related to temporary file creation coming from the entrypoint script loading the correct conda environment.
"Unsuccessfully failed" here means that the full experiment run was marked as good despite that error happening which in fact did not even allow for a proper start of the execution procedure.
The fix for this situation is to add a minimal writable tmpfs for /tmp in order to have the space necessary for that file to be created.
It might also be useful to have one for /run even if it does not look like it is being used currently (at least in the same way as /tmp seemed to not be used).
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/30
Error when waiting on a loop that wasn't started
2020-02-06T17:36:09Z
Samuel GAIST
Error when waiting on a loop that wasn't started
If for some reason the loop, or database, `process` method was not called and `wait` is called, a runtime error will occur because the message handler was not started as expected by the current code.
If for some reason the loop, or database, `process` method was not called and `wait` is called, a runtime error will occur because the message handler was not started as expected by the current code.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/93
New environment descovery does not respect the raise on error property
2020-01-29T13:15:19Z
Samuel GAIST
New environment descovery does not respect the raise on error property
If the discovery process using image labels is run on a machine without docker running or if the daemon is not running, it will fail as expected however the error handling does not follow the raise on error property.
To reproduce, call ...
If the discovery process using image labels is run on a machine without docker running or if the daemon is not running, it will fail as expected however the error handling does not follow the raise on error property.
To reproduce, call `beat editor refresh-env` on a machine without docker or with the daemon stopped.
A current workaround is to create the `.environments.json` file in the prefix with `{}` as its content.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/90
Execute experiment in different environment
2019-10-07T13:31:09Z
Samuel GAIST
Execute experiment in different environment
Currently, when executing an experiment on a local machine, the current environment is used and therefor must contain all the packages needed.
This is sub-optimal as several different experiences might required different sets of package...
Currently, when executing an experiment on a local machine, the current environment is used and therefor must contain all the packages needed.
This is sub-optimal as several different experiences might required different sets of packages.
While this can't be done with the local executor, it can be implemented using the subprocess executor.
The subprocess executor makes call to the scripts needed to run the algorithms/databases processes as would be with the docker executor. This fact can be used to run the scripts from another environment and thus will use the packages available there.
This will also simplify the creation of new execution environment and their transition do docker images.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/27
Improve loop evaluator synchronized write
2019-10-01T10:08:15Z
Samuel GAIST
Improve loop evaluator synchronized write
The loop evaluator write method is call once per output write. However, a processor block may have several outputs which means that the evaluator write method will be called as many times as an output is written. So in the case of a sequ...
The loop evaluator write method is call once per output write. However, a processor block may have several outputs which means that the evaluator write method will be called as many times as an output is written. So in the case of a sequential processor algorithm with 2 outputs and 3 input data, the evaluator write method will be call 6 times.
After talking with @andre.anjos, the current solution proposed is to add the output name written to as a parameter of the evaluator write method so that the developer can chose on which one he wants to "synchronize" the evaluator output.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/26
Validation fails with utf-8 error
2019-09-25T09:25:30Z
Samuel GAIST
Validation fails with utf-8 error
Under some circumstances the values sent for validation may trigger an "utf-8" decoding error.
This comes from the fact that all zmq packets received are decoded before being passed to the callbacks corresponding to the received command...
Under some circumstances the values sent for validation may trigger an "utf-8" decoding error.
This comes from the fact that all zmq packets received are decoded before being passed to the callbacks corresponding to the received command.
The solution here is to pass the received data as is to the callbacks and they are responsible for decoding the data if needed.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/87
Remove parameters from analyzer block
2019-08-19T19:15:28Z
Samuel GAIST
Remove parameters from analyzer block
Currently analyzer blocks are treated mostly like normal algorithm blocks except the handling of the output.
This means that they can also have parameters. After several discussions, it has been concluded that having the possibility to ...
Currently analyzer blocks are treated mostly like normal algorithm blocks except the handling of the output.
This means that they can also have parameters. After several discussions, it has been concluded that having the possibility to parametrize an analyzer would bring an unwelcomed level of complexity especially when creating reports.
If an analyzer should provide different outputs, then it should come in several versions to make it clear from the start what it will provide and how the analysis is done.
The following decisions have been made:
- Keep the current few analyzers that do have parameters as they are
- Remove the parameters field from the V2 algorithm schema for the analyzer definition
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/86
Loop parts name
2019-07-31T08:30:04Z
Samuel GAIST
Loop parts name
Based on experience and discussions, the loop parts name are not really clear.
They will be changed to the following to make things clearer:
- XXX_loop_user -> XXX_loop_processor
- XXX_loop -> XXX_loop_evaluator
This will require upda...
Based on experience and discussions, the loop parts name are not really clear.
They will be changed to the following to make things clearer:
- XXX_loop_user -> XXX_loop_processor
- XXX_loop -> XXX_loop_evaluator
This will require updates to the corresponding algorithms.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/85
Synchronised input for loop blocks
2019-07-30T12:56:03Z
Samuel GAIST
Synchronised input for loop blocks
Currently the two algorithm making a loop macro block are not synchronized input wise.
This is required in sequential mode so the validating part of the loop knows it uses
the same data as the current processing part.
This issue tracks...
Currently the two algorithm making a loop macro block are not synchronized input wise.
This is required in sequential mode so the validating part of the loop knows it uses
the same data as the current processing part.
This issue tracks that input synchronization.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/84
Implement output for loop block
2019-07-30T07:15:14Z
Samuel GAIST
Implement output for loop block
Currently the loop blocks don't have any outputs.
This issue tracks the implementation of this feature in the loop block.
One requested constraint for these outputs is that the write to them should happen when a write occurs in the out...
Currently the loop blocks don't have any outputs.
This issue tracks the implementation of this feature in the loop block.
One requested constraint for these outputs is that the write to them should happen when a write occurs in the output of a loop user block.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/82
Improve duplicate key handling
2019-06-19T05:13:48Z
Samuel GAIST
Improve duplicate key handling
This issue tracks the implementation of duplicated key handling in JSON files used.
In normal situation, the files should be edited and stored using beat/beat.editor> or currently still beat/beat.web> however if one does edit the declar...
This issue tracks the implementation of duplicated key handling in JSON files used.
In normal situation, the files should be edited and stored using beat/beat.editor> or currently still beat/beat.web> however if one does edit the declaration file by hand, errors like duplicated key might be introduced which won't be caught as loading a JSON file line by line will result in a dictionary with the last entry being used (which is normal behavior when dealing with dictionary type of containers).
In our case, the system will generate an error so that nothing will run with unexpected values.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/25
Handle duplicate key in json data
2019-06-17T05:48:34Z
Samuel GAIST
Handle duplicate key in json data
Currently loading a json file that contains a key multiple times will result in the last entry being used.
To avoid getting strange result, implement a hook that will raise an error and stop there.
Currently loading a json file that contains a key multiple times will result in the last entry being used.
To avoid getting strange result, implement a hook that will raise an error and stop there.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/24
Assert usage cleanup
2019-05-10T08:16:51Z
Samuel GAIST
Assert usage cleanup
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.backend.python.
Relates to beat/beat.core#72
Following bandit warning about usage of assert in code, this issue is used to track down the cleanup of these statements found in beat.backend.python.
Relates to beat/beat.core#72
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/23
Improve storage classes
2019-05-08T08:53:02Z
Samuel GAIST
Improve storage classes
The Storage and CodeStorage classes share a parallel goals, part of their code is exactly the same yet they are two distinct classes.
Fix this by creating a common base class so they share the configuration member variable and their check.
The Storage and CodeStorage classes share a parallel goals, part of their code is exactly the same yet they are two distinct classes.
Fix this by creating a common base class so they share the configuration member variable and their check.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/79
Implement missing prototypes
2019-05-07T10:47:47Z
Samuel GAIST
Implement missing prototypes
Some assets currently don't have prototypes and therefor cannot be created from scratch.
This issue tracks the implementation for them.
Currently needed:
- [x] Database
- [x] Experiment
Note that experiment can't be created from scrat...
Some assets currently don't have prototypes and therefor cannot be created from scratch.
This issue tracks the implementation for them.
Currently needed:
- [x] Database
- [x] Experiment
Note that experiment can't be created from scratch so this use case is unsupported and should properly triggers an exception.
An experiment is based on a toolchain. Therefor providing a "prototype" doesn't make sense as its content would depend on what is available in the prefix. Also the person getting that "prototype" would first have to clear everything before it can start configuring the experiment as they would like.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/78
Plotterparameter saving is not operational
2019-05-06T13:35:08Z
Samuel GAIST
Plotterparameter saving is not operational
Currently the implementation of the writing of a Plotterparameter can succeed.
The implementation is wrong and there are elements of the class that are missing.
Currently the implementation of the writing of a Plotterparameter can succeed.
The implementation is wrong and there are elements of the class that are missing.
Soft loops
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.core/-/issues/59
Tight "soft-loops" controlling algorithmic training
2020-11-04T17:09:18Z
André Anjos
Tight "soft-loops" controlling algorithmic training
In the context of the ALLIES project, a so-far-avoided requirement has popped up again the meeting: block loops.
I created this bug to differentiate from #29, because the requirements here are a bit different: in the use-case to be cons...
In the context of the ALLIES project, a so-far-avoided requirement has popped up again the meeting: block loops.
I created this bug to differentiate from #29, because the requirements here are a bit different: in the use-case to be considered, platform users need to be able to train a model depending on the output of another block. The main research question of the project needs to answer is: is it possible to implement life-long learning systems (LLLS) that can be taught by a human and improve over time? The outcome of the project must be reproducible (therefore our involvement).
In order to simulate this, partners provided the following diagram:
![modes](/uploads/a4e1aa953cd2bced7fa834da5d300e63/modes.png)
On the left, you'll see conventional ways to "adapt" statistical models from new input data. Given the model and more data, the model can be adapted to new incoming information (a.k.a. "source" in these diagrams).
In the context of ALLIES, partners want to have reproducible toolchains that allow third-parties to create the training system on the right of this image. In this context, the yellow boxes under the "training" (cyan) boxes simulate a human-user interaction with the system. E.g.: it could be a human telling a machine translation system the translation of a sentence (or paragraph) needs to be corrected, alongside with the correction, or it could be a human telling a speaker diarization system that a certain segment belongs to a different speaker. Because humans have limited ability to provide corrections, the consortium wants to control the implementation and limits of the "yellow" boxes, while third-parties implement the "trainers" that go into the cyan boxes.
It is possible to "unroll" these loops for practical scenarios, but in order to test limits of LLLS, one can envisage hundreds if not thousand iterations which is not very practical to code in BEAT as of today.
One of the key questions while considering #29 was that of the cache: a system that loops will issue multiply different cache outputs which, using the current architecture, would have to be kept. In this new use-case, such a caching requirement is not present. Partners only want to ensure that the code in "yellow" boxes can be properly tagged by the consortium and that they can conduct searches and build leaderboards on experiments using those (and only those). This prevents implementors of the "cyan" training boxes to misinterpret the role and data from the simulated human (yellow boxes). As a consequence, **no caching is required within this loop**.
Soft loops
https://gitlab.idiap.ch/beat/beat.editor/-/issues/221
Open asset related files
2019-10-04T15:12:53Z
Samuel GAIST
Open asset related files
Files related to the asset are currently not shown nor editable through beat/beat.editor>.
Add the possibility to edit these using the user's corresponding default editor.
Files related to the asset are currently not shown nor editable through beat/beat.editor>.
Add the possibility to edit these using the user's corresponding default editor.
[v2] 1 - Edition/Visualization for small editors
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.editor/-/issues/183
Database Editor
2019-10-04T15:12:53Z
Jaden DIEFENBAUGH
Database Editor
[v2] 1 - Edition/Visualization for small editors
Samuel GAIST
Samuel GAIST
https://gitlab.idiap.ch/beat/beat.editor/-/issues/74
insert entire protocol based on a protocol template
2018-08-07T18:14:41Z
Jaden DIEFENBAUGH
insert entire protocol based on a protocol template
Miscellaneous Editor-specific Improvements