beat.core issueshttps://gitlab.idiap.ch/beat/beat.core/-/issues2017-08-06T11:17:04Zhttps://gitlab.idiap.ch/beat/beat.core/-/issues/21Output method write() is not able to process beat.core.Data instances2017-08-06T11:17:04ZManuel GüntherOutput method write() is not able to process beat.core.Data instancesThe output method write() is currently not able to process beat.core.Data instances.
This means that when a complex data structure is read from the inputs and that this input has to be written in one of the outputs, the user currently...The output method write() is currently not able to process beat.core.Data instances.
This means that when a complex data structure is read from the inputs and that this input has to be written in one of the outputs, the user currently has to convert it into a JSON like python dictionary.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/20Reliability of data I/O2017-08-06T11:17:04ZAndré AnjosReliability of data I/OSimilarly to what we did for the indexes, we should also promote some check-summing capabilities for the data which is produced and consumed in our environments. I leave this open for discussion here.Similarly to what we did for the indexes, we should also promote some check-summing capabilities for the data which is produced and consumed in our environments. I leave this open for discussion here.Second BEAT ReviewLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.core/-/issues/19More robust JSON validation2017-08-06T11:17:04ZAndré AnjosMore robust JSON validationWe currently have in place a hand-crafted JSON validation strategy for the many JSON bits we manipulate inside the platform.
I propose we improve JSON validation by introducing [JSON schemas](http://json-schema.org/) for every piece o...We currently have in place a hand-crafted JSON validation strategy for the many JSON bits we manipulate inside the platform.
I propose we improve JSON validation by introducing [JSON schemas](http://json-schema.org/) for every piece of JSON we handle in the platform. There is a [Python package available on PyPI](https://pypi.python.org/pypi/jsonschema) that is compatible with draft-4 (the latest) of the specification.
Find [here](http://spacetelescope.github.io/understanding-json-schema/), a gentle introduction to JSON schemas.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/18beat experiment pull does not pull all dataformats and complains2017-08-06T11:17:04ZAndré Anjosbeat experiment pull does not pull all dataformats and complainsAfter changes with the version numbers, a simple `beat experiments pull` for a particular experiment does not pull all dataformats required by the experiment anymore. As a result, the experiment cannot be run immediately after.After changes with the version numbers, a simple `beat experiments pull` for a particular experiment does not pull all dataformats required by the experiment anymore. As a result, the experiment cannot be run immediately after.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/17Refer to "ISO/IEC 19795-1:2006 "Biometric performance testing and reporting -...2018-11-03T15:23:12ZLaurent EL SHAFEYRefer to "ISO/IEC 19795-1:2006 "Biometric performance testing and reporting -- Part 1: Principles and framework" in performance evaluationFollowing a remark raised after the RP Conference, it would be nice to implement the biometric metrics according to the ISO standard. In particular, it seems that the ROC displayed during the conference was not respecting the definition ...Following a remark raised after the RP Conference, it would be nice to implement the biometric metrics according to the ISO standard. In particular, it seems that the ROC displayed during the conference was not respecting the definition given in the standard.
One simple option would be to add a reference to the standard in the documentation of the analyzer algorithms.https://gitlab.idiap.ch/beat/beat.core/-/issues/16beat experiment run does not use external environment2018-11-03T15:23:12ZAndré Anjosbeat experiment run does not use external environmentCurrently, the command to run experiments on the user machine does not rely on the same infrastructure used by the platform to run experiments.
It would be good these two things converged at a certain point.Currently, the command to run experiments on the user machine does not rely on the same infrastructure used by the platform to run experiments.
It would be good these two things converged at a certain point.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/15Pull experiment does not work properly for user 'tutorial'2018-11-03T15:23:12ZLaurent EL SHAFEYPull experiment does not work properly for user 'tutorial'When trying to pull experiments:
```sh
../bin/beat exp pull --user tutorial
```
we end up with:
```text
Pulling the experiment tutorial/eigenface/Test17
The toolchain isn't valid, due to the following errors:
Errors w...When trying to pull experiments:
```sh
../bin/beat exp pull --user tutorial
```
we end up with:
```text
Pulling the experiment tutorial/eigenface/Test17
The toolchain isn't valid, due to the following errors:
Errors with the algorithm 'tutorial/postperf': Failed to find the algorithm file
```Biometrics Center Kickoff Meeting and BEAT ReviewLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.core/-/issues/14createDataFromJSON() is failing on arrays of arrays structures2018-11-03T15:23:12ZLaurent EL SHAFEYcreateDataFromJSON() is failing on arrays of arrays structuresThe function createDataFromJSON() is failing on arrays of array structures, such as:
```
{
"input_subtract": "beat/array_1d_floats",
"input_divide": "beat/array_1d_floats",
"weights": [
0,
"beat/a...The function createDataFromJSON() is failing on arrays of array structures, such as:
```
{
"input_subtract": "beat/array_1d_floats",
"input_divide": "beat/array_1d_floats",
"weights": [
0,
"beat/array_2d_floats"
],
"biases": [
0,
"beat/array_1d_floats"
]
}
```
This currently generates the error:
*** ValueError: field named value not found.
where 'value' is the key of the 'beat/array_*' dictionary.https://gitlab.idiap.ch/beat/beat.core/-/issues/13Input management2018-11-03T15:23:12ZAndré AnjosInput managementIt would be interesting to revisit input management on algorithms and analysis blocks and try to factor out code that is repeated everywhere into a simpler interface.
This bug shall be used to track advances on this field.It would be interesting to revisit input management on algorithms and analysis blocks and try to factor out code that is repeated everywhere into a simpler interface.
This bug shall be used to track advances on this field.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/12Object renaming2018-11-03T15:23:12ZAndré AnjosObject renamingIt should be possible to rename:
- [ ] algorithms
- [ ] dataformats
- [ ] toolchains
- [ ] experiments
The platform should handle renaming gracefully.It should be possible to rename:
- [ ] algorithms
- [ ] dataformats
- [ ] toolchains
- [ ] experiments
The platform should handle renaming gracefully.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/11(Local) Paced development2018-08-02T06:30:28ZAndré Anjos(Local) Paced developmentWhen the user develops a bit toolchain, s/he cannot test the toolchain if algorithms for *all* blocks are not implemented. This makes development a bit hard and demotivating.
A simple solution to this problem would be to implement som...When the user develops a bit toolchain, s/he cannot test the toolchain if algorithms for *all* blocks are not implemented. This makes development a bit hard and demotivating.
A simple solution to this problem would be to implement some sort of "noop" algorithm that could live inside the "beat" user account.
In local execution mode (with `beat run`), if such an algorithm would be found, then it would not execute the current algorithm or any other following it. This would allow the user to place "beat/noop" as an algorithm in all blocks to be developed and focus on a single block at a time.https://gitlab.idiap.ch/beat/beat.core/-/issues/10Library support2018-11-03T15:23:12ZAndré AnjosLibrary supportA ticket for holding information on library support. Some aspects discussed over the last months:
- The easiest to start with would be: 1 library = 1 file, pretty much like algorithms.
- External (bigger packages) need to be installe...A ticket for holding information on library support. Some aspects discussed over the last months:
- The easiest to start with would be: 1 library = 1 file, pretty much like algorithms.
- External (bigger packages) need to be installed with the processing environment.
- Libraries should be version-nable such as algorithms.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/9Versionning support2018-11-03T15:23:12ZAndré AnjosVersionning supportAutomatic versionning suport should be implemented through out the platform. I'm putting this bug here, but surely this affects most packages in the project. I detail our existing workplan and some key behavioral aspects.
- Each objec...Automatic versionning suport should be implemented through out the platform. I'm putting this bug here, but surely this affects most packages in the project. I detail our existing workplan and some key behavioral aspects.
- Each object in the system (databases, toolchains, dataformats, algorithms, attached documentation strings, environments) should be version-able. Every time the user uploads or changes that object, the version number of that object is bumped.
- Each time a new version is entered, the user should be able to attach a version comment, which is independent from the object documentation itself.
- For the platform, the combination of the object name and the version determines the uniqueness of such object.
- The sharing, usage and tagging status of each unique object (name/version) should be tracked independently. This outdates the usage of "/private" paths which seems to be a bit confusing for some users.
- While visualizing the object, the user by defaults accesses the latest version to which it has access to. This should be made consistent through the platform.
- There is always an option to see/use an earlier version of an object, but that requires the user to specify it explicitly. For example, in the command-line utilities, a "--version=xxx" could be passed. On the web interface, the version of a certain object is chosen automatically as the latest, but a field allows the user to browse for an earlier version.Biometrics Center Kickoff Meeting and BEAT Reviewhttps://gitlab.idiap.ch/beat/beat.core/-/issues/8Old scripts lingering at beat git2014-09-22T14:36:37ZAndré AnjosOld scripts lingering at beat gitI just realized that the following scripts are still available on `./bin`:
- generate_single_integers.py
- check_dataformat.py
- synchronize.py
- run_toolchain.py
- generate_integers_database.py
- check_toolchain.py
- display_da...I just realized that the following scripts are still available on `./bin`:
- generate_single_integers.py
- check_dataformat.py
- synchronize.py
- run_toolchain.py
- generate_integers_database.py
- check_toolchain.py
- display_data.py
If they are not useful anymore (i.e. have been integrated into the `beat` general command-line program), could you please remove them not to create any confusion?Biometrics Center Kickoff Meeting and BEAT ReviewAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/7Improvements to the beat command-line tool (migrated from github)2015-03-25T07:37:09ZAndré AnjosImprovements to the beat command-line tool (migrated from github)- [x] Pull a configuration
- [x] Push a configuration
- [x] Edition/Pull/Push of descriptions associated to different objects
- [x] Color configuration
- [x] Check of a toolchain `beat toolchain check ...`
- [x] Algorithm changes de...- [x] Pull a configuration
- [x] Push a configuration
- [x] Edition/Pull/Push of descriptions associated to different objects
- [x] Color configuration
- [x] Check of a toolchain `beat toolchain check ...`
- [x] Algorithm changes detection when running an experiment
- [x] Check user existence in commands like: `beat algorithms pull --user ADHDAHGDGAHSGHDAGHSDGH`. Presently, this is silently ignored.
- [x] While running locally, it would be nice that this package printed out more information about data input and output. It is currently very summarized.
Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/6Support for enumerations (migrated from github)2014-09-22T14:36:49ZAndré AnjosSupport for enumerations (migrated from github)I have not seen how to declare an enumeration. Is this possible at all? We do have use cases where we could benefit from this. E.g., client data migth belong to the "train", "devel" or "test" datasets. These could be implemented as enume...I have not seen how to declare an enumeration. Is this possible at all? We do have use cases where we could benefit from this. E.g., client data migth belong to the "train", "devel" or "test" datasets. These could be implemented as enumerations. Currently, in the UBM-GMM, I implement this field as an unchecked string.
Biometrics Center Kickoff Meeting and BEAT ReviewPhilip ABBETPhilip ABBEThttps://gitlab.idiap.ch/beat/beat.core/-/issues/5Plotting library request (migrated from github)2014-09-05T12:58:54ZAndré AnjosPlotting library request (migrated from github)It would be useful if the platform provided a set of library functions to allow the creation of charts in easier terms, instead of everyone re-implementing theirs. Bob already provides APIs for plotting. It would make sense that BEAT pro...It would be useful if the platform provided a set of library functions to allow the creation of charts in easier terms, instead of everyone re-implementing theirs. Bob already provides APIs for plotting. It would make sense that BEAT provided a similar API for functionality in Bob:
A good starting point for this API is Bob's plotting API, [which is available here](http://www.idiap.ch/software/bob/docs/nightlies/last/bob/sphinx/html/measure/index.html).
A tricky plot to perform is a DET curve. I hope this is still possible using RGraph's API.https://gitlab.idiap.ch/beat/beat.core/-/issues/4Binary data format (migrated from github)2018-07-30T08:57:48ZAndré AnjosBinary data format (migrated from github)The current data format of choice uses the Python pickling system. This is highly subjective to the project module structure. I'm not sure it is a good idea to keep it hanging around for long.
Our group had a (very good) experience wi...The current data format of choice uses the Python pickling system. This is highly subjective to the project module structure. I'm not sure it is a good idea to keep it hanging around for long.
Our group had a (very good) experience with HDF5. This is a binary format that is quite flexible, fast to read, compact and universally compatible with all major software suites. I believe we should port the data format
system to use data instead of Python pickle.
To get started, I'd go for incorporating a dependence on the excellent python package `h5py`. It does already bring support for all Python/NumPy basic types and is extensible to different data types.
At this point, it would be also nice to introduce data versioning somewhere. Maybe on the "dataformat" descriptors (not sure).
> Note: Philip/François are not sure on HDF5. Before discarding it the possibility to use it, it would be interesting to
> understand why.
> Note: Another useful tool would be an automatic converter, that takes HDF5 files and transforms that into JSON
> descriptors of that format. One can convert HDF5 files to simple data descriptors and, from there, into BEAT's
> JSON format if necessary.
> Note: While debugging, it should be possible to inspect the cache. We currently have no tools to do so,
> but if we address this item with HDF5, then this should come for free.
Second BEAT ReviewLaurent EL SHAFEYLaurent EL SHAFEYhttps://gitlab.idiap.ch/beat/beat.core/-/issues/3Repeated connection is not detected on toolchain JSON (migrated from github)2015-03-06T08:04:27ZAndré AnjosRepeated connection is not detected on toolchain JSON (migrated from github)By mistake, I created a repeated connection from one block to another, on my list of connections. The platform has signaled the toolchain to be valid, but when running, the following error occurred:
```python
File "/remote/filer.gx...By mistake, I created a repeated connection from one block to another, on my list of connections. The platform has signaled the toolchain to be valid, but when running, the following error occurred:
```python
File "/remote/filer.gx/user.active/aanjos/work/projects/beat/beat/eggs/ipdb-0.7-py2.7.egg/ipdb/__main__.py", line 138, in main
pdb._runscript(mainpyfile)
File "/usr/lib/python2.7/pdb.py", line 1233, in _runscript
self.run(statement)
File "/usr/lib/python2.7/bdb.py", line 400, in run
exec cmd in globals, locals
File "<string>", line 1, in <module>
File "./bin/run_toolchain.py", line 3, in <module>
import sys
File "/remote/filer.gx/user.active/aanjos/work/projects/beat/beat/src/beat.core/beat/core/scripts/run_toolchain.py", line 186, in main
if isinstance(connection.output.block, Database):
AttributeError: 'NoneType' object has no attribute 'output'
Uncaught exception. Entering post mortem debugging
```
I guess that should be relatively easy to fix.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/issues/2Warning for unconnected/multiply connected blocks (migrated from github)2015-03-06T08:05:11ZAndré AnjosWarning for unconnected/multiply connected blocks (migrated from github)Along the same lines as #39, it would be convenient that both `check_toolchain.py` and `run_toolchain.py` would check for unconnected blocks and issue a **warning** in case one is detected. It would be an immense helper for debugging new...Along the same lines as #39, it would be convenient that both `check_toolchain.py` and `run_toolchain.py` would check for unconnected blocks and issue a **warning** in case one is detected. It would be an immense helper for debugging new toolchains.Tutorial for the FG2015 ConferenceAndré AnjosAndré Anjos