beat.core issueshttps://gitlab.idiap.ch/beat/beat.core/-/issues2014-08-28T11:34:02Zhttps://gitlab.idiap.ch/beat/beat.core/-/issues/1createDataFromJSON() is failing with dataformats which are arrays of dictiona...2014-08-28T11:34:02ZLaurent EL SHAFEYcreateDataFromJSON() is failing with dataformats which are arrays of dictionaries of dictionariesIt seems that the method createDataFromJSON() from the DataFormatSerializer does not work well when we have a dataformat, which is a list of dictionaries of dictionaries.
E.g.:
```
{
"objs": [2,
{
"id": "int...It seems that the method createDataFromJSON() from the DataFormatSerializer does not work well when we have a dataformat, which is a list of dictionaries of dictionaries.
E.g.:
```
{
"objs": [2,
{
"id": "int32",
"coordinates": {"x": "int32",
"y": "int32"
}
}
]
}
```
In this case, the method returns None and the associated error string is:
Data instance has no attribute 'coordinates'
I've pushed a 'failing' unit test in the repository (b8d4d1bc3 and e489e67f5)https://gitlab.idiap.ch/beat/beat.core/-/issues/5Plotting library request (migrated from github)2014-09-05T12:58:54ZAndré AnjosPlotting library request (migrated from github)It would be useful if the platform provided a set of library functions to allow the creation of charts in easier terms, instead of everyone re-implementing theirs. Bob already provides APIs for plotting. It would make sense that BEAT pro...It would be useful if the platform provided a set of library functions to allow the creation of charts in easier terms, instead of everyone re-implementing theirs. Bob already provides APIs for plotting. It would make sense that BEAT provided a similar API for functionality in Bob:
A good starting point for this API is Bob's plotting API, [which is available here](http://www.idiap.ch/software/bob/docs/nightlies/last/bob/sphinx/html/measure/index.html).
A tricky plot to perform is a DET curve. I hope this is still possible using RGraph's API.https://gitlab.idiap.ch/beat/beat.core/-/issues/8Old scripts lingering at beat git2014-09-22T14:36:37ZAndré AnjosOld scripts lingering at beat gitI just realized that the following scripts are still available on `./bin`:
- generate_single_integers.py
- check_dataformat.py
- synchronize.py
- run_toolchain.py
- generate_integers_database.py
- check_toolchain.py
- display_da...I just realized that the following scripts are still available on `./bin`:
- generate_single_integers.py
- check_dataformat.py
- synchronize.py
- run_toolchain.py
- generate_integers_database.py
- check_toolchain.py
- display_data.py
If they are not useful anymore (i.e. have been integrated into the `beat` general command-line program), could you please remove them not to create any confusion?Biometrics Center Kickoff Meeting and BEAT ReviewAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/6Support for enumerations (migrated from github)2014-09-22T14:36:49ZAndré AnjosSupport for enumerations (migrated from github)I have not seen how to declare an enumeration. Is this possible at all? We do have use cases where we could benefit from this. E.g., client data migth belong to the "train", "devel" or "test" datasets. These could be implemented as enume...I have not seen how to declare an enumeration. Is this possible at all? We do have use cases where we could benefit from this. E.g., client data migth belong to the "train", "devel" or "test" datasets. These could be implemented as enumerations. Currently, in the UBM-GMM, I implement this field as an unchecked string.
Biometrics Center Kickoff Meeting and BEAT ReviewPhilip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.core/-/issues/3Repeated connection is not detected on toolchain JSON (migrated from github)2015-03-06T08:04:27ZAndré AnjosRepeated connection is not detected on toolchain JSON (migrated from github)By mistake, I created a repeated connection from one block to another, on my list of connections. The platform has signaled the toolchain to be valid, but when running, the following error occurred:
```python
File "/remote/filer.gx...By mistake, I created a repeated connection from one block to another, on my list of connections. The platform has signaled the toolchain to be valid, but when running, the following error occurred:
```python
File "/remote/filer.gx/user.active/aanjos/work/projects/beat/beat/eggs/ipdb-0.7-py2.7.egg/ipdb/__main__.py", line 138, in main
pdb._runscript(mainpyfile)
File "/usr/lib/python2.7/pdb.py", line 1233, in _runscript
self.run(statement)
File "/usr/lib/python2.7/bdb.py", line 400, in run
exec cmd in globals, locals
File "<string>", line 1, in <module>
File "./bin/run_toolchain.py", line 3, in <module>
import sys
File "/remote/filer.gx/user.active/aanjos/work/projects/beat/beat/src/beat.core/beat/core/scripts/run_toolchain.py", line 186, in main
if isinstance(connection.output.block, Database):
AttributeError: 'NoneType' object has no attribute 'output'
Uncaught exception. Entering post mortem debugging
```
I guess that should be relatively easy to fix.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/2Warning for unconnected/multiply connected blocks (migrated from github)2015-03-06T08:05:11ZAndré AnjosWarning for unconnected/multiply connected blocks (migrated from github)Along the same lines as #39, it would be convenient that both `check_toolchain.py` and `run_toolchain.py` would check for unconnected blocks and issue a **warning** in case one is detected. It would be an immense helper for debugging new...Along the same lines as #39, it would be convenient that both `check_toolchain.py` and `run_toolchain.py` would check for unconnected blocks and issue a **warning** in case one is detected. It would be an immense helper for debugging new toolchains.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/7Improvements to the beat command-line tool (migrated from github)2015-03-25T07:37:09ZAndré AnjosImprovements to the beat command-line tool (migrated from github)- [x] Pull a configuration
- [x] Push a configuration
- [x] Edition/Pull/Push of descriptions associated to different objects
- [x] Color configuration
- [x] Check of a toolchain `beat toolchain check ...`
- [x] Algorithm changes de...- [x] Pull a configuration
- [x] Push a configuration
- [x] Edition/Pull/Push of descriptions associated to different objects
- [x] Color configuration
- [x] Check of a toolchain `beat toolchain check ...`
- [x] Algorithm changes detection when running an experiment
- [x] Check user existence in commands like: `beat algorithms pull --user ADHDAHGDGAHSGHDAGHSDGH`. Presently, this is silently ignored.
- [x] While running locally, it would be nice that this package printed out more information about data input and output. It is currently very summarized.
Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/44Agent is unprotected in case the user sends to many "nxt" commands2017-08-06T11:17:03ZAndré AnjosAgent is unprotected in case the user sends to many "nxt" commandsWe should put in place:
1. A check in the agent side (around line 150), that prevents the agent to measure the length of `None'
2. A new protocol reply possibility for an erroneous condition on the `nxt` operation.
We should put in place:
1. A check in the agent side (around line 150), that prevents the agent to measure the length of `None'
2. A new protocol reply possibility for an erroneous condition on the `nxt` operation.
BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/42[io] Execute externally currently requires twisted (async.run)2017-08-06T11:17:03ZLaurent EL SHAFEY[io] Execute externally currently requires twisted (async.run)With the introduction of the new I/O framework, the execution of a job requires to launch the I/O daemon (that will launch and control the user process). This daemon currently needs twisted (async.run) to be executed, while we would idea...With the introduction of the new I/O framework, the execution of a job requires to launch the I/O daemon (that will launch and control the user process). This daemon currently needs twisted (async.run) to be executed, while we would ideally prefer not to have such a dependency in beat.core.
To fix this, an optimal decapsulation of the communication via pipes from the I/O daemon would be required, as well as its integration into beat.core.BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/41[io] Optimization of I/O when using pipes2017-08-06T11:17:03ZLaurent EL SHAFEY[io] Optimization of I/O when using pipesWith the introduction of communication via pipes between the user process and the daemon, it may happen that a chunk of data is decoded from a baseformat and then re-encoded (into a baseformat) to send it through pipe. This is the case, ...With the introduction of communication via pipes between the user process and the daemon, it may happen that a chunk of data is decoded from a baseformat and then re-encoded (into a baseformat) to send it through pipe. This is the case, when next() is called: a chunk of data is then read using a CachedDataSource (that will decode from a baseformat) and then sent through a pipe (and hence will be re-encoded into a baseformat). This is of course suboptimal and may be fixed by specializing the CachedDataSource.BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/40Cleanup of executor/dataformats/algorithms/libraries/schema loading2017-08-06T11:17:03ZAndré AnjosCleanup of executor/dataformats/algorithms/libraries/schema loadingAs of today, we moved the definition of these objects and functions to beat.backend.python. The backend does not need to be able to validate objects, only to load them and instantiate a runner. We should revise and clean-up this relation...As of today, we moved the definition of these objects and functions to beat.backend.python. The backend does not need to be able to validate objects, only to load them and instantiate a runner. We should revise and clean-up this relationship.BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/39inputs/outputs cleanup w.r.t beat.backend.python2017-08-06T11:17:03ZAndré Anjosinputs/outputs cleanup w.r.t beat.backend.pythonAs of today there is some functionality of beat.backend.python that is duplicated in this package. We should clean-up and organised it so there is no repetition of code, respecting a good DRY principle.As of today there is some functionality of beat.backend.python that is duplicated in this package. We should clean-up and organised it so there is no repetition of code, respecting a good DRY principle.BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/38discover_environments() does not crash when the same environment/version is d...2017-08-06T11:17:03ZLaurent EL SHAFEYdiscover_environments() does not crash when the same environment/version is detected multiple timesWhile this should raise an exception.While this should raise an exception.BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/35Move command-line client to another package2017-08-06T11:17:03ZAndré AnjosMove command-line client to another packageAs discussed, we should move the command-line client to another package, since it depends on the `beat.web` API. This creates some sort of reverse dependence which we should get rid of.As discussed, we should move the command-line client to another package, since it depends on the `beat.web` API. This creates some sort of reverse dependence which we should get rid of.BEAT TourAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/34JSON declarations should be versioned2017-08-06T11:17:03ZAndré AnjosJSON declarations should be versionedWe should include a version field on the JSON declaration to allow for changes on the declaration to co-exist.We should include a version field on the JSON declaration to allow for changes on the declaration to co-exist.BTAS TutorialAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/32Dash '-' as a valid (or invalid) character for element names (dataformats, al...2017-08-06T11:17:03ZLaurent EL SHAFEYDash '-' as a valid (or invalid) character for element names (dataformats, algorithms, etc.)?I've noticed an inconsistency wrt. the usage of the dash '-' character for element names.
There is currently a bunch of elements (e.g. algorithm 'ptome/fingervein-miura_scoring/1/') defined on the platform, which contains the '-' char...I've noticed an inconsistency wrt. the usage of the dash '-' character for element names.
There is currently a bunch of elements (e.g. algorithm 'ptome/fingervein-miura_scoring/1/') defined on the platform, which contains the '-' character.
In contrast, the '-' character is not 'always' part of the valid character at the JSON validation schema level (see beat/core/schema/experiment.json line 48).
The list of valid characters needs to be properly defined, and the characters accepted by the beat.web interface need to match the ones of the JSON validation schema.BEAT TourLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.core/-/issues/31Experiment setup() does not handle properly blocks with multiple outputs2017-08-06T11:17:03ZLaurent EL SHAFEYExperiment setup() does not handle properly blocks with multiple outputsFollowing the major revamping of this package, blocks with multiple outputs are not handled in the right way.
In particular, the current implementation of the setup() method for Experiment assumes that a block has a single output (e.g. ...Following the major revamping of this package, blocks with multiple outputs are not handled in the right way.
In particular, the current implementation of the setup() method for Experiment assumes that a block has a single output (e.g. see https://gitlab.idiap.ch/biometric/beat.core/blob/v0.8.1/beat/core/experiment.py#L858).
I am unsure whether there are other implications in the code or not.Tutorial for the FG2015 ConferenceLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.core/-/issues/30beat db index improvements2017-08-06T11:17:03ZAndré Anjosbeat db index improvementsShould index all databases, if not specified which.
It should also support multiple databases to be indexed.Should index all databases, if not specified which.
It should also support multiple databases to be indexed.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/28No way to visualise (binary) cache data2017-08-06T11:17:03ZAndré AnjosNo way to visualise (binary) cache dataAfter our data re-structuring in this package, a useful feature was removed: the ability to write cache files as JSON text. Cache files are now written in binary format, which improves the calculation precision and translates to what hap...After our data re-structuring in this package, a useful feature was removed: the ability to write cache files as JSON text. Cache files are now written in binary format, which improves the calculation precision and translates to what happens inside the platform. It is also much faster.
A side effect was we lost the possibility to directly view cache files.
A command-line option/command at our `beat` swiss-army knife could allow us to recover this ability, by providing a "view" command that allows one to inspect the contents of binary cache files.
This bug will remain opened until this is fixed.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/27Downloading/Uploading of database declarations and views2017-08-06T11:17:03ZAndré AnjosDownloading/Uploading of database declarations and viewsOnce beat.web#86 is fixed, then we should implement the pull/push of database declarations via the command-line as well.Once beat.web#86 is fixed, then we should implement the pull/push of database declarations via the command-line as well.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjos