# This is a combination of 2 commits.

# This is the 1st commit message:

[sphinx] Redoing the documentation

# This is the commit message #2:

[sphinx] Documenting

doc/baseline.rst

-.. _bob.bio.base.baseline:
-
-==================
-Defining baselines
-==================
-
-
-Once you have a biometric system well established, tuned and working for a
-particular database (or a particular set of databases), you may want to provide
-**an easier to reproduce** way to share it. For this purpose, we defined
-something called baseline.
-
-A baseline (:any:`bob.bio.base.baseline.Baseline`) is composed by the triplet
-of :any:`bob.bio.base.preprocessor.Preprocessor`,
-:any:`bob.bio.base.extractor.Extractor` and
-:any:`bob.bio.base.algorithm.Algorithm`.
-
-First, check it out the baselines ready to be triggered in your environment by
-doing:
-
-.. code-block:: sh
-
-    $ bob bio baseline --help
-
-For example, if you run ``bob bio baseline -vvv eigenface atnt``, it will run
-the eigenface face recognition baseline on the atnt database (assuming you have
-installed ``bob.bio.face`` and ``bob.db.atnt``).
-
-
-To create your own baseline, you just need to define it like in the recipe
-below:
-
-.. code-block:: py
-
-    from bob.bio.base.baseline import Baseline
-
-    baseline = Baseline(name="my-baseline",
-                        preprocessors={"default": 'my-preprocessor'},
-                        extractor='my-extractor'),
-                        algorithm='my-algorithm'))
-
-Some databases may require some specific preprocessors depending on the type
-of meta-informations provided. For instance, for some face recognition
-databases, faces should be cropped in a particular way depending on the
-annotations provided. To approach this issue, the preprocessors are defined in
-a dictionary, with a generic preprocessor defined as **default** and the
-database specific preprocessor defined by database name as in the example
-below:
-
-.. code-block:: py
-
-    self.preprocessors = dict()
-    self.preprocessors["default"] = 'my-preprocessor'
-    self.preprocessors["database_name"] = 'my-specific-preprocessor'
-
-
-Follow below a full example on how to define a baseline with database specific
-preprocessors.
-
-.. code-block:: py
-
-    from bob.bio.base.baseline import Baseline
-
-    preprocessors = {"default": 'my-preprocessor'}
-    preprocessors["database_name"] = 'my-specific-preprocessor'
-    baseline = Baseline(name="another-baseline",
-                        preprocessors=preprocessors,
-                        extractor='my-extractor'),
-                        algorithm='my-algorithm'))
-
-.. note::
-
-   The triplet can be a resource or a configuration file. This works in the
-   same way as in :ref:`Running Experiments <running_part_1>`.
-
-.. note::
-
-  Baselines are also registered as resources under the keyword
-  `bob.bio.baseline`.
-
-You can find the list of readily available baselines using the ``resources.py``
-command:
-
-.. code-block:: sh
-
-    $ resources.py --types baseline

doc/index.rst

@@ -38,24 +38,6 @@ Here is a growing list of derived packages:
 * :ref:`bob.bio.video <bob.bio.video>` An extension of face recognition algorithms to run on video data, and the according video databases
 * :ref:`bob.bio.gmm <bob.bio.gmm>` Algorithms based on Gaussian Mixture Modeling (GMM) such as Inter-Session Variability modeling (ISV) or Total Variability modeling (TV, aka. I-Vector) [Pri07]_ and [ESM+13]_.
 
-Tutorial on the Biometric Recognition Framework
-===============================================
-
-As an introduction into the Biometric Recognition Framework, a.k.a., bob.bio, we presented a tutorial in collaboration with the International Joint Conference on Biometrics (IJCB) in 2017.
-The tutorial was called *Bob’s Biometric Recognition Framework - A Hands-on Tutorial with Face Recognition Examples* and introduced the bob.bio framework by showing how to use and how to extend the framework.
-Though all presented examples were about face recognition, other biometric modalities can be handled similarly.
-
-If you did not have the time to attend the tutorial itself, we recorded the session.
-You can find the videos on YouTube, either combined in my `YouTube Channel <https://www.youtube.com/playlist?list=PL-M6OZEM9v0FXfgvarnrpTMhURafZbXwp>`__, or separately here:
-
-1. `Introduction <https://youtu.be/rYOgKqjWarw>`__
-2. `Bob's Biometric Framework <https://youtu.be/YaWBNpTTko8>`__
-3. `Gaussian Mixture Models <https://www.youtube.com/watch?v=YhWbpYh_-QY>`__
-4. `Extending the Biometric Framework <https://www.youtube.com/watch?v=6zcc-KCAK-8>`__
-5. `Parenthesis <https://www.youtube.com/watch?v=4IK_L-dVT3Y>`__
-6. `Reproducible Research <https://www.youtube.com/watch?v=0rVCkUoRVE0>`__
-
-If you want to test out the tutorial, you might want to have a look at its `Corresponding Web Page <http://vast.uccs.edu/public-data/IJCB.html>`__.
 
 Citing our Publications
 =======================
@@ -99,7 +81,6 @@ Users Guide
    struct_bio_rec_sys
    experiments
    implementation
-   baseline
    filelist-guide
    more
    annotations

doc/installation.rst

@@ -44,8 +44,7 @@ command line below will install all the required packages:
                     bob.bio.face \
                     bob.bio.gmm \
                     bob.bio.video \
-                    bob.db.youtube \
-                    gridtk
+                    bob.db.youtube
 
 
 Databases
@@ -58,32 +57,15 @@ data of the databases, please refer to the according Web-pages. For a list of
 supported databases including their download URLs, please refer to the
 `biometric recognition databases`_.
 
-After downloading the raw data for the databases, you will need to tell
-``bob.bio``, where these databases can be found. For this purpose, we have
-decided to implement a special file, where you can set your directories. By
-default, this file is located in ``~/.bob_bio_databases.txt``, and it contains
-several lines, each line looking somewhat like:
+After downloading the **raw data** for the databases, you will need to tell
+``bob.bio``, where these databases can be found. This can be set using
+our :ref:`bob.extension.rc`.
+For instance, the name convention to be followed in the `~/.bobrc` file is: `<database_package_name>.directory`. 
+The command below shows how to set the path of the :ref:`bob.db.youtube`.
 
-.. code-block:: text
-
-   [YOUR_ATNT_DIRECTORY] = /path/to/your/directory
-
-.. note::
-
-   If this file does not exist, feel free to create and populate it yourself.
-
-
-Please use ``databases.py`` for a list of known databases, where you can see
-the raw ``[YOUR_DATABASE_PATH]`` entries for all databases that you haven't
-updated, and the corrected paths for those you have.
-
-.. note::
+.. code-block:: sh
 
-   If you have installed only ``bob.bio.base``, there is no database listed --
-   as all databases are included in other packages, such as
-   :ref:`bob.bio.face <bob.bio.face>` or :ref:`bob.bio.spear <bob.bio.spear>`.
-   Also, please don't forget that you need to install the corresponding
-   ``bob.db.<name>`` package as well.
+   bob config set bob.db.youtube.directory /path/to/the/youtube/database
 
 
 Test your Installation
@@ -108,24 +90,6 @@ packages separately.
   $ nosetests -vs bob.bio.gmm
   ...
 
-Some of the tests that are run require the images of the `AT&T database`_
-database. If the database is not found on your system, it will automatically
-download and extract the `AT&T database`_ a temporary directory, **which will
-not be erased**.
-
-To avoid the download to happen each time you call the nose tests, please:
-
-1. Download the `AT&T database`_ database and extract it to the directory of
-   your choice.
-2. Set an environment variable ``ATNT_DATABASE_DIRECTORY`` to the directory,
-   where you extracted the database to. For example, in a ``bash`` you can
-   call:
-
-.. code-block:: sh
-
-  $ export ATNT_DATABASE_DIRECTORY=/path/to/your/copy/of/atnt
-
-
 In case any of the tests fail for unexplainable reasons, please send a report
 through our `mailing list`_.
 

doc/py_api.rst

@@ -21,6 +21,13 @@ IO-related functions
    bob.bio.base.vstack_features
 
 
+Pipelines
+~~~~~~~~~
+
+.. autosummary::
+   bob.bio.base.pipelines.vanilla_biometrics.biometric_pipeline
+
+
 Functions dealing with resources
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -42,58 +49,7 @@ Miscellaneous functions
    bob.bio.base.selected_indices
 
 
-Tools to run recognition experiments
-------------------------------------
-
-Command line generation
-~~~~~~~~~~~~~~~~~~~~~~~
-
-.. autosummary::
-   bob.bio.base.tools.command_line_parser
-   bob.bio.base.tools.initialize
-   bob.bio.base.tools.command_line
-   bob.bio.base.tools.write_info
-   bob.bio.base.tools.FileSelector
-
-Controlling of elements
-~~~~~~~~~~~~~~~~~~~~~~~
-
-.. autosummary::
-   bob.bio.base.tools.groups
-   bob.bio.base.tools.indices
-
-Preprocessing
-~~~~~~~~~~~~~
-
-.. autosummary::
-   bob.bio.base.tools.preprocess
-   bob.bio.base.tools.read_preprocessed_data
-
-Feature Extraction
-~~~~~~~~~~~~~~~~~~
-
-.. autosummary::
-   bob.bio.base.tools.train_extractor
-   bob.bio.base.tools.extract
-   bob.bio.base.tools.read_features
-
-Algorithm
-~~~~~~~~~
-
-.. autosummary::
-   bob.bio.base.tools.train_projector
-   bob.bio.base.tools.project
-   bob.bio.base.tools.train_enroller
-   bob.bio.base.tools.enroll
 
-Scoring
-~~~~~~~
-
-.. autosummary::
-   bob.bio.base.tools.compute_scores
-   bob.bio.base.tools.concatenate
-   bob.bio.base.tools.calibrate
-   bob.bio.base.script.figure.Metrics
 
 Loading data
 ------------
@@ -125,16 +81,6 @@ OpenBR conversions
 Details
 -------
 
-.. automodule:: bob.bio.base
-
-   .. attribute:: valid_keywords
-
-      Valid keywords, for which resources are defined, are ``('database', 'preprocessor', 'extractor', 'algorithm', 'grid')``
-
-
-.. automodule:: bob.bio.base.tools
-
-   .. autoclass:: FileSelector
 
 .. automodule:: bob.bio.base.score.load
 .. automodule:: bob.bio.base.score.openbr

doc/struct_bio_rec_sys.rst

@@ -13,40 +13,88 @@ On the other hand, an *identification* system attempts to establish a person's i
 
 A biometric recognition system has two stages:
 
-1. **Enrollment:** A person's biometric data is enrolled to the system's biometric database.
-2. **Recognition:** A person's newly acquired biometric data (which we call a *probe*) is compared to the enrolled biometric data (which we refer to as a *model*), and a match score is generated.  The match score tells us how similar the model and the probe are.  Based on match scores, we then decide whether or not the model and probe come from the same person (verification) or which gallery identity should to be assigned to the input biometric (identification).
+1. **Creation of biometric references:** A person's biometric data is registered to the system's biometric database.
+2. **Recognition:** A person's newly acquired biometric data (which we call a *probe*) is compared to a biometric reference (which we refer to as a *model*), and a match score is generated.  The match score tells us how similar the model and the probe are.  Based on match scores, we then decide whether or not the model and probe come from the same person (verification) or which gallery identity should to be assigned to the input biometric (identification).
 
-Fig. 1 shows the enrollment and verification stages in a typical biometric *verification* system:
+Fig. 1 shows the creation of biometric references and verification stages in a typical biometric *verification* system:
 
 .. figure:: img/bio_ver_sys.svg
    :align: center
 
-   Enrollment and verification in a typical biometric verification system.
+   Creation of biometric references and verification in a typical biometric verification system.
 
-Fig. 2 shows the enrollment and identification stages in a typical biometric *identification* system:
+Fig. 2 shows the creation of biometric references and identification stages in a typical biometric *identification* system:
 
 .. figure:: img/bio_ident_sys.svg
    :align: center
 
-   Enrollment and identification in a typical biometric identification system.
+   Creation of biometric references and identification in a typical biometric identification system.
 
 In the figures above:
 
 * The "Pre-processor" cleans up the raw biometric data to make recognition easier (e.g., crops the face image to get rid of the background).
 * The "Feature Extractor" extracts the most important features for recognition, from the pre-processed biometric data.
-* The "Model Database" stores each person's extracted feature set in the form of a representative model for that person in the system database, typically alongside the person's ID.
-* The "Matcher" compares a new biometric feature set (probe) to one (for verification) or all (for identification) models in the database, and outputs a similarity score for each comparison.
+* The "Biometric references" stores each person's extracted feature set in the form of a representative model for that person in the system database, typically alongside the person's ID.
+* In the "Comparison" stage a new biometric feature set (probe) to one (for verification) or all (for identification) models in the database, and outputs a similarity score for each comparison.
 * For *verification*, the "Decision Maker" decides whether or not the probe and the model from the database match, based on whether the similarity score is above or below a pre-defined match threshold.  For *identification*, the "Decision Maker" decides which model from the database best represents the identity of the probe, based on which model most closely matches the probe.
 
 
 Biometric Recognition Experiments in the ``bob.bio`` Framework
 ---------------------------------------------------------------
 
-The ``bob.bio`` framework has the capability to perform both *verification* and *identification* experiments, depending on the user's requirements.  To talk about the framework in generic terms, we will henceforth use the term *recognition*.
+
+
+The ``bob.bio`` framework has the capability to perform both *verification* and *identification* experiments, depending on the user's requirements.
+To talk about the framework in generic terms, we will henceforth use the term *recognition*.
 
 In general, the goal of a biometric recognition experiment is to quantify the recognition accuracy of a biometric recognition system, e.g., we wish to find out how good the system is at deciding whether or not two biometric samples come from the same person.
 
-To conduct a biometric recognition experiment, we need biometric data.  So, we use a biometric database.  A biometric database generally consists of multiple samples of a particular biometric, from multiple people.  For example, a face database could contain 5 different images of a person's face, from 100 people.  The dataset is split up into samples used for enrollment, and samples used for probing.  We enroll a model for each identity from one or more of its faces. We then simulate "genuine" recognition attempts by comparing each person's probe samples to their enrolled models.  We simulate "impostor" recognition attempts by comparing the same probe samples to models of different people. Which data is used for training, enrollment and probing is defined by the evaluation *protocol* of the database. The protocol also defines, which models should be compared to which probes.
+Figures 1 and 2 described pseudo-pipelines on how a biometric recognition system works.
+In fact, each one of those figures describe two sub-pipelines.
+One that creates **biometric references** given input data; and one that performs **biometric recognition** given input data and biometric references.
+Those two sub-pipelines are implemented in ``bob.bio`` using `Dask <https://dask.org/>` delayed.
+Check it out its documentation on how to create pipelines using Dask.
+Furthermore, bob.pipelines provides some examples on how to build pipelines using dask and on how to deploy them in different computational clusters.
+
+
+
+
+Sub pipeline 1 - Creation of biometric references
+*************************************************
+
+This sub-pipeline is composed by, at least, three elements in the following order: i-) `Preprocess`, ii-) `Feature extraction` and iii-
+
+
+Sub pipeline 2 - Biometric Recognition
+**************************************
+
+Optional - Sub pipeline 3 - Training background model
+*****************************************************
+
+
+
+
+AN ADITIONAL
+
+
+To conduct a biometric recognition experiment, we need biometric data.
+Hence, we use a biometric database.
+A biometric database generally consists of multiple samples of a particular biometric, from multiple people.
+For example, a face database could contain 5 different images of a person's face, from 100 people.
+The dataset is split up into samples used for enrollment, and samples used for probing.
+We enroll a model for each identity from one or more of its faces.
+We then simulate "genuine" recognition attempts by comparing each person's probe samples to their enrolled models.
+We simulate "impostor" recognition attempts by comparing the same probe samples to models of different people.
+Which data is used for training, enrollment and probing is defined by the evaluation *protocol* of the database.
+The protocol also defines, which models should be compared to which probes.
+
+
+In section above it was described how a biometric s 
+
+
+Stages
+------
+
 
 In ``bob.bio``, biometric recognition experiments are split up into four main stages, similar to the stages in a typical verification or identification system as illustrated in Fig. 1 and Fig. 2, respectively:
 
@@ -58,7 +106,7 @@ In ``bob.bio``, biometric recognition experiments are split up into four main st
 Each of these stages is discussed below:
 
 Data Preprocessing:
-~~~~~~~~~~~~~~~~~~~
+*******************
 
 Biometric measurements are often noisy, containing redundant information that is not necessary (and can be misleading) for recognition.  For example, face images contain non-face background information, vein images can be unevenly illuminated, speech signals can be littered with background noise, etc.  The aim of the data preprocessing stage is to clean up the raw biometric data so that it is in the best possible state to make recognition easier.  For example, biometric data is cropped from the background, the images are photometrically enhanced, etc.
 
@@ -71,7 +119,7 @@ All the biometric samples in the input biometric database go through the preproc
 
 
 Feature Extraction:
-~~~~~~~~~~~~~~~~~~~
+*******************
 
 Although the preprocessing stage produces cleaner biometric data, the resulting data is usually very large and still contains much redundant information.  The aim of the feature extraction stage is to extract features that are necessary for recognizing a person.
 
@@ -87,7 +135,7 @@ All the biometric features stored in the "preprocessed" directory go through the
 
 
 Matching:
-~~~~~~~~~
+*********
 
 The matching stage in ``bob.bio`` is referred to as the "Algorithm".  The Algorithm stage consists of three main parts:
 
@@ -122,7 +170,7 @@ The matching stage in ``bob.bio`` is referred to as the "Algorithm".  The Algori
 
 
 Decision Making:
-~~~~~~~~~~~~~~~~
+****************
 
 The decision making stage in ``bob.bio`` is referred to as "Evaluation".  If we wish to perform *verification*, then the aim of this stage will be to make a decision as to whether each score calculated in the Matching stage indicates a "Match" or "No Match" between the particular model and probe biometrics.  If we wish to perform *identification*, then the aim of the evaluation stage will be to find the model which most closely matches the probe biometric.