beat.core merge requestshttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests2020-09-02T11:18:10Zhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/127Update docker related information2020-09-02T11:18:10ZSamuel GAISTUpdate docker related informationFollowing the release of BEAT runtime images on Docker hub and beat/beat.env.examples>, update the documentation related to the use of Docker.Following the release of BEAT runtime images on Docker hub and beat/beat.env.examples>, update the documentation related to the use of Docker.André AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/126Pre commit cleanup2020-08-06T08:14:38ZSamuel GAISTPre commit cleanupThis merge request updates the pre-commit configuration and applies a full source cleanup.This merge request updates the pre-commit configuration and applies a full source cleanup.https://gitlab.idiap.ch/beat/beat.core/-/merge_requests/125Add minimal rw tmpfs for /tmp and /run2020-07-29T09:18:52ZSamuel GAISTAdd minimal rw tmpfs for /tmp and /runAt least in production, some issues happened that required
to have /tmp writable. To preserve the read-only state
of the container, use a small tmpfs that is enough for
the container to run. /run is another that might be needed
so make i...At least in production, some issues happened that required
to have /tmp writable. To preserve the read-only state
of the container, use a small tmpfs that is enough for
the container to run. /run is another that might be needed
so make it part of the defaults used.
Fixes #101Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/124Add test for setup in cxx algorithm2020-07-21T11:07:52ZSamuel GAISTAdd test for setup in cxx algorithmCurrently only the processing part during an experiment run was tested.
This new test ensures that the setup feature works properly.Currently only the processing part during an experiment run was tested.
This new test ensures that the setup feature works properly.André AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/123Ensure that writing to a non existing output shows an error2020-07-08T15:29:37ZSamuel GAISTEnsure that writing to a non existing output shows an errorThe test added by this merge request ensure that writing to an output that does not exists is reported.
The error will be that "NoneType" as no property "write" which is expected following how outputs work.
Part of #99The test added by this merge request ensure that writing to an output that does not exists is reported.
The error will be that "NoneType" as no property "write" which is expected following how outputs work.
Part of #99Soft loopsSamuel GAISTSamuel GAISThttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/122Add optionnal environment entry to database2020-06-26T10:11:18ZSamuel GAISTAdd optionnal environment entry to databaseThis merge request add support for defining the database environment to use in the database declaration.
This allows the BEAT infrastructure to no depend on list of database supported by a given environment and allows for more flexibili...This merge request add support for defining the database environment to use in the database declaration.
This allows the BEAT infrastructure to no depend on list of database supported by a given environment and allows for more flexibility to test new databases locally especially when the use of Docker is required when running for example C++ algorithms.
Part of #64Soft loopshttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/121Fix database prototype2020-06-18T13:58:35ZSamuel GAISTFix database prototypeThe database prototype setup method has a wrong signature and is used in beat/beat.editor> when creating a new Database asset.
This merge request fixes it as well as add the call to the default implementation which is mandatory for ever...The database prototype setup method has a wrong signature and is used in beat/beat.editor> when creating a new Database asset.
This merge request fixes it as well as add the call to the default implementation which is mandatory for everything to be setup properly.
The setup method has been added to two Views in the test prefix so that its use is also tested.
Fixes #97Soft loopshttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/120Move to new images2020-06-09T08:09:09ZSamuel GAISTMove to new imagesThis merge request moves all the test images to the new version created through beat/beat.env.builder>
The use of these images means that the old image enumeration becomes effectively a backup solution.
The test should now take less ti...This merge request moves all the test images to the new version created through beat/beat.env.builder>
The use of these images means that the old image enumeration becomes effectively a backup solution.
The test should now take less time as the label based search is faster.
Using these new images also means that we can archive all the various beat.env.XXX repositories and only use beat/beat.env.builder>.Soft loopsAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/119Add test assets with too long descriptions2020-03-13T11:04:08ZSamuel GAISTAdd test assets with too long descriptionsThis will allow to improve tests also in beat/beat.cmdline>
Related to beat/beat.cmdline#65This will allow to improve tests also in beat/beat.cmdline>
Related to beat/beat.cmdline#65Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/118Logger cleanup2020-03-11T13:44:30ZSamuel GAISTLogger cleanupThis merge request update all logger.warn calls to logger.warning as warn is obsolete.
It also removes all unused logger instances.This merge request update all logger.warn calls to logger.warning as warn is obsolete.
It also removes all unused logger instances.Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/117Add size constraint for description field2020-03-12T16:01:52ZSamuel GAISTAdd size constraint for description fieldThis merge request adds a size constraint to ensure that the description field is not longer than what is allowed in beat/beat.web>
Fixes #95
Related to beat/beat.editor#255This merge request adds a size constraint to ensure that the description field is not longer than what is allowed in beat/beat.web>
Fixes #95
Related to beat/beat.editor#255Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/116Don't use f string yet2020-03-11T07:55:55ZSamuel GAISTDon't use f string yetThe processing nodes are not on Python 3.6 yet
therefore these can not be used.
The other uses are only related to tests and
tasks not done on the processing nodes.The processing nodes are not on Python 3.6 yet
therefore these can not be used.
The other uses are only related to tests and
tasks not done on the processing nodes.Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/115Only enable GPU runtime if available2020-03-04T16:39:56ZSamuel GAISTOnly enable GPU runtime if availableSome execution environments may run CPU as well as GPU
code, this check for Nvidia driver presence will allow
to reuse the same environment on workers with and without
GPU available.
It also updates the GPU option to use the latest vers...Some execution environments may run CPU as well as GPU
code, this check for Nvidia driver presence will allow
to reuse the same environment on workers with and without
GPU available.
It also updates the GPU option to use the latest version
provided by the nvidia-docker2 documentation.Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/114Improve execution handling2020-03-03T13:12:35ZSamuel GAISTImprove execution handlingThis merge request enforces the use of the current user for the execution of the docker container where nothing specific was written in the configuration.
This also allows to make the container itself read-only.
Part of #94This merge request enforces the use of the current user for the execution of the docker container where nothing specific was written in the configuration.
This also allows to make the container itself read-only.
Part of #94Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/113Add missing variable initialisation for optional argument in worker script2020-02-25T08:35:56ZSamuel GAISTAdd missing variable initialisation for optional argument in worker scriptThe variable initialisation was missing when calling the script without using the `--docker` option.The variable initialisation was missing when calling the script without using the `--docker` option.Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/112Improve disconnection handling2020-02-20T16:18:56ZSamuel GAISTImprove disconnection handlingWhen reloading a worker node, the broker will notice after the usual timeout that it was gone. This is the same as if a node loses its network connection or gets killed.
This wasn't an issue before because the restart of the worker scri...When reloading a worker node, the broker will notice after the usual timeout that it was gone. This is the same as if a node loses its network connection or gets killed.
This wasn't an issue before because the restart of the worker script was "slow" due to the enumeration of all BEAT related docker images. If a node uses the new labeled images, the restart is way faster and thus the broker will have an invalid state.
This patch fixes that by sending a notification to the broker that the worker node is disconnecting. This will make the broker cleanup the entry and have it ready once the worker is back online.Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/111improve complex setup handling2020-02-20T14:28:26ZSamuel GAISTimprove complex setup handlingThis patch implements support for using NFS mounts for cache entries.
Depending on how the cache is made accessible to the worker machines, a direct mount might not be possible. This patch fixes that by using NFS in the same manner as i...This patch implements support for using NFS mounts for cache entries.
Depending on how the cache is made accessible to the worker machines, a direct mount might not be possible. This patch fixes that by using NFS in the same manner as it is done for the databases access.
The downside of this is that the NFS mounts can only be done for folders and not individual files like standard local volumes.
Part of #94Soft loopsFlavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/110Make cache access more specific2020-02-07T15:16:13ZSamuel GAISTMake cache access more specificThis improves the handling of the cache by only mounting what is necessary.
Part of #94This improves the handling of the cache by only mounting what is necessary.
Part of #94Soft loopsAndré AnjosAndré Anjoshttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/109Improve handling of message handler shutdown in local executor2020-02-07T08:33:37ZSamuel GAISTImprove handling of message handler shutdown in local executorIf one of the message handler was not started before joined, it would raise an error.
See [Python.threading.Thread.join](https://docs.python.org/3/library/threading.html#threading.Thread.join)
Relates to beat/beat.backend.python#30If one of the message handler was not started before joined, it would raise an error.
See [Python.threading.Thread.join](https://docs.python.org/3/library/threading.html#threading.Thread.join)
Relates to beat/beat.backend.python#30Flavio TARSETTIFlavio TARSETTIhttps://gitlab.idiap.ch/beat/beat.core/-/merge_requests/108Handle docker exception following the raise on error property for the new env...2020-01-29T13:15:18ZSamuel GAISTHandle docker exception following the raise on error property for the new environment discoveryIF docker is installed or the daemon not accessible, the exception
propagation will not follow the raise on error property. This patch
fixes that and thus keeps the behaviour consistant.
Fixes #93IF docker is installed or the daemon not accessible, the exception
propagation will not follow the raise on error property. This patch
fixes that and thus keeps the behaviour consistant.
Fixes #93Soft loopsAndré AnjosAndré Anjos