guide.rst

.. vim: set fileencoding=utf-8 :

.. Copyright (c) 2016 Idiap Research Institute, http://www.idiap.ch/          ..
.. Contact: beat.support@idiap.ch                                             ..
..                                                                            ..
.. This file is part of the beat.web module of the BEAT platform.             ..
..                                                                            ..
.. Commercial License Usage                                                   ..
.. Licensees holding valid commercial BEAT licenses may use this file in      ..
.. accordance with the terms contained in a written agreement between you     ..
.. and Idiap. For further information contact tto@idiap.ch                    ..
..                                                                            ..
.. Alternatively, this file may be used under the terms of the GNU Affero     ..
.. Public License version 3 as published by the Free Software and appearing   ..
.. in the file LICENSE.AGPL included in the packaging of this file.           ..
.. The BEAT platform is distributed in the hope that it will be useful, but   ..
.. WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY ..
.. or FITNESS FOR A PARTICULAR PURPOSE.                                       ..
..                                                                            ..
.. You should have received a copy of the GNU Affero Public License along     ..
.. with the BEAT platform. If not, see http://www.gnu.org/licenses/.          ..


.. _overview:

==========
 Overview
==========

The |project| platform can be used to explore and reproduce experiments in
machine learning and pattern recognition. One of major goals of the |project|
platform is re-usability and reproducibility of research experiments. The
|project| platform uses several mechanisms to ensure that each experiment is
reproducible and experiments share parts with each other as much as possible.
For example in a simple experiment, the database, the algorithm, and the
environment used to run the experiment can be shared between experiments.

A fundamental part of the :ref:`Experiments` in the |project| platform is a
toolchain. You can see an example of a toolchain below:

.. image:: img/toolchain.*


:ref:`Toolchains` are sequences of blocks and links (like a block diagram)
which represent the data flow on an experiment. Once you have defined the
toolchain against which you'd like to run, the experiment can be further
configured by assigning different datasets, algorithms and analyzers to the
different toolchain blocks.

The data that circulates at each toolchain connection in a configured
experiment is formally defined using :ref:`Dataformats`. The platform
experiment configuration system will prevent you from using incompatible
data formats between connecting blocks. For example, in the toolchain depicted
above, if one decides to use a dataset on the block ``train`` (top-left of the
diagram) that yields images, the subsequent block cannot contain an algorithm
that can only treat audio signals. Once the dataset on the block ``train`` has
been chosen, the platform will not offer you incompatible algorithms to be
placed on the following regular block ``linear_machine_training`` - only
*compatible* algorithms.

You can create your own experiments and develop your own algorithms or change
existing ones. An experiment usually consists of three main bits: the
toolchain, datasets, and algorithms.

- The datasets are *provided* system resources offered by the |project|
  platform instance you are using. Normal users cannot create or change
  available datasets in the platform (see our :ref:`faq`). Please refer to
  :ref:`faq` for more information.

- You can learn about how to develop new algorithms or change existing ones by
  looking at our :ref:`Algorithms` section. A special kind of algorithms are
  result *analyzers* that are used to generate the results of the experiments.

You can check the results of an experiment once it finishes and share the
results of your experiments with other people or :ref:`teams` using our sharing
interface. Moreover, you can request for attestation (refer to
:ref:`attestations`) of your experiment which ensures it reproducible and that
its details cannot be changed anymore. This is done by locking the resources
that were used in your experiment.

:ref:`reports` are like *macro*-:ref:`attestations`. You can lock several
experiments together generate a selected list of tables and figures you can
re-use on your scientific publications. Reports can be used to gather results
of different experiments of one particular publication in one page so that you
can share it with others with a single link.

Experiments and almost any other resources can be copied (or *forked* in
|project|-parlance) ensure maximum re-usability and reproducibility. Users can
fork different resources change them and use them in their own experiments. In
the :ref:`searchlabel` section you can learn about how you can search for
different resources available to you on the |project| platform.

You can also use our search feature to create *leaderboards*. Leaderboards are
stored searches of experiment results with certain criteria and filters. The
experiment results are sorted by one of the available criteria of your choosing
(e.g. a performance indicator that you consider valuable). Leaderboards are
regularly updated by the platform machinery and, as soon as it changes, you
will be notified (consult our section on :ref:`usersettings` for information on
how to enable or disable e-mail notifications). This helps you to keep track of
your favorite machine learning or pattern recognition tasks and datasets.