Commit 5aeb187c authored by Samuel GAIST's avatar Samuel GAIST

[algorithms] Add information about data loader

parent 89a23257
......@@ -41,11 +41,39 @@ defined at a higher-level in the platform. It is expected that the
implementation of the algorithm respects whatever was declared on the
platform.
By default, the algorithm is **data-driven**; algorithm is typically provided
.. _beat-system-algorithms-types:
Algorithm types
---------------
Previous versions of the beat.core package implemented only one type of
algorithm (referred to as v1 algorithm). The current version of beat.core
defines two type of algorithms:
- Sequential
- Autonomous
The sequential algorithm type is the direct successor of the v1 algorithm. For
migration information, see :ref:`_beat-system-algorithms-api-migration`.
Sequential
..........
The sequential algorithm is **data-driven**; algorithm is typically provided
one data sample at a time and must immediately produce some output data.
Autonomous
..........
The autonomous algorithm as its name suggest is responsible for loading the data
samples it needs in order to do its work. He's also responsible for writing the
appropriate amount of data on its outputs.
Furthermore, the way the algorithm handle the data is highly configurable and
covers a huge range of possible scenarios.
:numref:`beat-system-overview-block` displays the relationship between a
processing block and its algorithm.
......@@ -123,9 +151,8 @@ from the other groups. This is described in more details in the following
subsections. Finally, the field `description` is optional and gives a short
description of the algorithm.
The graphical interface of BEAT provides editor for algorithm,
which simplifies its `JSON`_ declaration definition. It also includes a simple
Python code editor.
The web client of the BEAT platform provides a graphical editor for algorithm,
which simplifies its `JSON`_ declaration definition.
.. _beat-system-algorithms-definition-analyzer:
......@@ -137,8 +164,8 @@ At the end of the processing workflow of an experiment, there is a special
kind of algorithm, which does not yield any output, but in contrast so called
`results`. These algorithms are called **analyzers**.
`Results` of an experiment are reported back to the user. Data privacy is very
important in the BEAT system and therefore only a limited number of data formats can be
`Results` of an experiment are reported back to the user. Since the platform
is concerned about data privacy, only a limited number of data formats can be
employed as results in an analyzer, such as boolean, integers, floating point
values, strings (of limited size), as well as plots (such as scatter or bar
plots).
......@@ -197,7 +224,7 @@ generates an ROC curve as well as few other metrics.
Source Code
...........
The BEAT system has been designed to support algorithms written in different
The BEAT platform has been designed to support algorithms written in different
programming languages. However, for each language, a corresponding back-end
needs to be implemented, which is in charge of connecting the inputs and
outputs to the algorithm and running its code as expected. In this section,
......@@ -213,10 +240,10 @@ conventions. In the following, examples of such classes are provided.
Examples
--------
.. _beat-system-algorithms-examples-simple:
.. _beat-system-algorithms-examples-simple-sequential:
Simple algorithm (no parametrization)
.....................................
Simple sequential algorithm (no parametrization)
................................................
At the very minimum, an algorithm class must look like this:
......@@ -224,7 +251,7 @@ At the very minimum, an algorithm class must look like this:
class Algorithm:
def process(self, inputs, outputs):
def process(self, inputs, data_loaders, outputs):
# Read data from inputs, compute something, and write the result
# of the computation on outputs
...
......@@ -232,21 +259,51 @@ At the very minimum, an algorithm class must look like this:
The class must be called ``Algorithm`` and must have a method called
``process()``, that takes as parameters a list of inputs (see section
:ref:`beat-system-algorithms-input-inputlist`) and a list of outputs (see
section :ref:`beat-system-algorithms-output-outputlist`). This method must
:ref:`beat-system-algorithms-input-inputlist`), a list of data loader (see section
:ref:`beat-system-algorithms-dataloader-dataloaderlist`) and a list of outputs
(see section :ref:`beat-system-algorithms-output-outputlist`). This method must
return ``True`` if everything went correctly, and ``False`` if an error
occurred.
The platform will call this method once per block of data available on the
`synchronized` inputs of the block.
.. _beat-system-algorithms-examples-simple-autonomous:
.. _beat-system-algorithms-examples-parametrizable:
Simple autonomous algorithm (no parametrization)
................................................
At the very minimum, an algorithm class must look like this:
.. code-block:: python
class Algorithm:
def process(self, data_loaders, outputs):
# Read data from data_loaders, compute something, and write the
# result of the computation on outputs
...
return True
The class must be called ``Algorithm`` and must have a method called
``process()``, that takes as parameters a list of data loader (see section
:ref:`beat-system-algorithms-dataloaders`) and a list of outputs (see
section :ref:`beat-system-algorithms-output-outputlist`). This method must
return ``True`` if everything went correctly, and ``False`` if an error
occurred.
The platform will call this method only once as it is its responsibility to load
the appropriate amount of data and process it.
.. _beat-system-algorithms-examples-parameterizable:
Parameterizable algorithm
.........................
........................
The following is valid for all types of algorithms
To implement a parameterizable algorithm, two things must be added to the class:
To implement a parametrizable algorithm, two things must be added to the class:
(1) a field in the JSON declaration of the algorithm containing their default
values as well as the type of the parameters, and (2) a method called
``setup()``, that takes one argument, a map containing the parameters of the
......@@ -274,7 +331,7 @@ algorithm.
self.threshold = parameters['threshold']
return True
def process(self, inputs, outputs):
def process(self, inputs, data_loaders, outputs):
# Read data from inputs, compute something, and write the result
# of the computation on outputs
...
......@@ -284,6 +341,36 @@ When retrieving the value of the parameters, one must not assume that a value
was provided for each parameter. This is why we may use a *try: ... except: ...*
construct in the ``setup()`` method.
.. _beat-system-algorithms-preparation:
Preparation of an algorithm
...........................
The following is valid for all types of algorithms
Often algorithms need to compute some values or retrieve some data prior to
applying their mathematical logic.
This is possible using the prepare method.
.. code-block:: python
class Algorithm:
def prepare(self, data_loaders):
data_loader = data_loaders.loaderOf('in2')
(data, _, _) = data_loader[0]
self.offset = data['in2'].value
return True
def process(self, inputs, data_loaders, outputs):
# Read data from inputs, compute something, and write the result
# of the computation on outputs
...
return True
.. _beat-system-algorithms-input:
Handling input data
......@@ -340,7 +427,7 @@ Each input provides the following informations:
.. py:attribute:: data_index
*(integer)* Index of the last block of data received on the input (See section
:ref:`beat-core-algorithms-input-synchronization`)
:ref:`beat-system-algorithms-input-synchronization`)
.. py:attribute:: data
......@@ -366,7 +453,7 @@ not in the same order. The two blocks use different algorithms, which both
refers to their inputs and outputs using names of their choice
Nevertheless, Joe can choose to use Bill's algorithm instead of his own one.
When the algorithm to use is changed on the graphical interface, the system will
When the algorithm to use is changed on the web interface, the platform will
attempt to match each input with the names (and types) declared by the
algorithm. In case of ambiguity, the user will be asked to manually resolve it.
......@@ -561,6 +648,51 @@ This will be addressed in a later version.
Feedback loop
.. _beat-system-algorithms-dataloaders:
Data loaders
------------
.. _beat-system-algorithms-dataloaders-dataloaderlist:
DataLoader list
...............
An algorithm is given accessto the **list of data loaders** of the processing
block. This list can be used to access each data loader individually, either by
their channel name (see :ref:`beat-system-algorithms-dataloaders-name`), their
index or by iterating over the list:
.. code-block:: python
# 'data_loaders' is the list of data loaders of the processing block
# Retrieve a data loader by name
data_loader = data_loaders['labels']
# Retrieve a data loader by index
for index in range(0, len(data_loaders)):
data_loader = data_loaders[index]
# Iteration over all data loaders
for data_loader in data_loaders:
...
# Retrieve the data loader an input belongs to, by input name
data_loader = data_loaders.loaderOf('label')
.. _beat-system-algorithms-dataloaders-dataloader:
DataLoader
..........
Provides access to data from a group of inputs synchronized together.
See .. :py:class::DataLoader:
.. _beat-system-algorithms-output:
Handling output data
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment