Merge branch 'update' into 'master'

Add back the tutorial

See merge request !27

doc/help.rst

@@ -32,7 +32,7 @@ There are several ways to get help if you are facing a problem.
 First, you should search for possible existing answers in:
 
 * https://stackoverflow.com/questions/tagged/python-bob
-* https://www.idiap.ch/software/bob/discuss
+* https://www.idiap.ch/software/bob/discuss (read only mailing list)
 * https://www.idiap.ch/software/bob/wiki
 
 or you may just want to search on the Internet for possible answers.
@@ -48,11 +48,7 @@ How to contribute
 =================
 
 Bob is open source and we welcome contributions.
-If you find a bug, please let use know through our mailing list:
-https://www.idiap.ch/software/bob/discuss.
-If you want to contribute, please also get in touch with us through our mailing list
-first.
-We maintain a mirror of Bob packages on https://github.com/bioidiap/ which should allow
-you to open pull requests on Bob packages.
+We maintain a mirror of Bob packages on https://github.com/bioidiap/.
+Feel free to open issues and/or pull requests on this mirror.
 
 .. include:: links.rst

doc/index.rst

@@ -10,11 +10,11 @@ Bob_ is a free signal-processing and machine learning toolbox originally develop
 the Biometrics Security and Privacy Group, the Biosignal Processing Group, and the
 Research and Development Engineers at `Idiap`_ Research Institute, Switzerland.
 
-The toolbox is written in a mix of `Python`_ and `C++`_ and is designed to be both
-efficient and reduce development time. It is composed of a reasonably large number of
-:ref:`independent packages <bob.packages>` that implement tools for image, audio &
-video processing, machine learning & pattern recognition, and a lot more task specific
-packages.
+The newest version of the toolbox is written in pure `Python`_ and is designed
+to be both efficient and reduce development time. It is composed of seveal
+:ref:`independent packages <bob.packages>` that implement tools for image, audio
+& video processing, machine learning & pattern recognition, and a lot more task
+specific packages.
 
 
 .. toctree::
@@ -23,6 +23,7 @@ packages.
    install
    help
    packages
+   tutorial
 
 
 Index of all packages
@@ -37,6 +38,7 @@ Index of all packages
    bob.bio.spear <bob/bob.bio.spear/doc/index.rst>
    bob.bio.vein <bob/bob.bio.vein/doc/index.rst>
    bob.bio.video <bob/bob.bio.video/doc/index.rst>
+   bob.devtools <bob/bob.devtools/doc/index.rst>
    bob.extension <bob/bob.extension/doc/index.rst>
    bob.fusion.base <bob/bob.fusion.base/doc/index.rst>
    bob.io.base <bob/bob.io.base/doc/index.rst>

doc/install.rst

@@ -8,32 +8,35 @@ By now you should know that Bob is made of several :ref:`bob.packages`. There is
 no single package that installs all Bob packages because that would just take
 too much space. Follow the instruction below to install any Bob package.
 
-We offer pre-compiled binary installations of Bob using `conda`_ for Linux and
-MacOS 64-bit operating systems. Follow the guide below to learn to install any
-Bob package. Bob does not work on Windows.
+We offer conda_ and pip_ installations of Bob for Linux and MacOS 64-bit
+operating systems (Most packages support arm 64-bit on MacOS too). Follow the
+guide below to learn to install any Bob package. *Bob does not work on
+Windows*.
 
-#.  Install `mamba`_  (`mambaforge`_ is preferred) or `conda`_ (`miniforge`_ is
+#.  Install `mamba`_  (`mambaforge`_ is preferred) or conda_ (`miniforge`_ is
     preferred) and get familiar with it. The instructions below use ``mamba``
     because it's faster than ``conda``, but you may replace it by ``conda``.
 
-#.  Make sure you have an up-to-date `conda`_ installation (conda 4.4 and above
+#.  Make sure you have an up-to-date conda_ installation (conda 4.4 and above
     is needed) with the **correct configuration** by running the commands below:
 
     .. code:: sh
 
        # install mamba if you don't yet have it installed.
+       # mamba must be installed in your base environment.
        $ conda install -n base -c conda-forge mamba
        $ mamba update -n base -c conda-forge conda mamba
        $ conda config --set show_channel_urls True
 
 #.  Create an environment with the specific Bob packages that you need. For
-    example if you want to install ``bob.io.base`` and ``bob.bio.face``:
+    example if you want to install ``bob.io.base``, ``bob.bio.face``, and ``pytorch``:
 
     .. code:: sh
 
        $ mamba create --name bob_env1 --override-channels \
-         -c https://www.idiap.ch/software/bob/conda -c conda-forge \
-         python=3 bob.io.base bob.bio.face
+         -c https://www.idiap.ch/software/bob/conda \
+         -c conda-forge \
+         python=3 bob.io.base bob.bio.face pytorch
 
 #.  Then activate the environment and configure its channels to make sure the
     channel list is correct in the future as well:
@@ -51,6 +54,13 @@ Bob package. Bob does not work on Windows.
        $ conda activate bob_env1
        $ mamba install bob.bio.video ...
 
+#. Alternatively, you can use pip_ to install the packages but we don't test our pip packages:
+
+    .. code:: sh
+
+       $ conda activate bob_env1
+       $ pip install bob.bio.face ...
+
 For a comprehensive list of packages that are either part of |project| or use
 |project|, please visit :ref:`bob.packages`.
 

doc/links.rst

@@ -9,15 +9,19 @@
 .. _c++: https://isocpp.org/
 .. _CMake: http://www.cmake.org
 .. _conda: https://conda.io/
+.. _dask: https://dask.org
 .. _Dvipng: http://savannah.nongnu.org/projects/dvipng/
 .. _FFMpeg: http://ffmpeg.org
 .. _fftw: http://www.fftw.org/
 .. _giflib: http://giflib.sourceforge.net/
 .. _GPL-2.0: http://www.opensource.org/licenses/GPL-2.0
 .. _GPL-3.0: http://www.opensource.org/licenses/GPL-3.0
+.. _h5py: http://www.h5py.org/
 .. _HDF5 License: ftp://ftp.hdfgroup.org/HDF5/current/src/unpacked/COPYING
 .. _HDF5: http://www.hdfgroup.org/HDF5
 .. _idiap: http://www.idiap.ch
+.. _imageio-ffmpeg: https://github.com/imageio/imageio-ffmpeg
+.. _imageio: https://imageio.readthedocs.io
 .. _install: https://www.idiap.ch/software/bob/install
 .. _IPython: http://ipython.scipy.org
 .. _Lapack: http://www.netlib.org/lapack
@@ -41,6 +45,7 @@
 .. _NumPy: http://www.numpy.org
 .. _packages: https://www.idiap.ch/software/bob/packages
 .. _Pillow: http://python-pillow.github.io/
+.. _pip: https://pip.pypa.io/en/stable/
 .. _pkg-config: http://www.freedesktop.org/wiki/Software/pkg-config/
 .. _Python-2.0: http://www.opensource.org/licenses/Python-2.0
 .. _python: http://www.python.org

doc/packages.rst

@@ -21,12 +21,6 @@ Machine Learning
 * :ref:`bob.learn.em`
 * :ref:`bob.pipelines`
 
-Modules for Developers
-----------------------
-
-* :ref:`bob.devtools`
-* :ref:`bob.extension`
-
 
 Biometric Recognition
 ---------------------
@@ -47,9 +41,8 @@ Presentation Attack Detection (anti-spoofing)
 * :ref:`bob.fusion.base`
 
 
-Database Interfaces
--------------------
+Modules for Developers
+----------------------
 
-* :ref:`bob.pipelines`
-* :ref:`bob.bio.base`
-* :ref:`bob.pad.base`
+* :ref:`bob.devtools`
+* :ref:`bob.extension`

doc/readme_index.rst

@@ -12,6 +12,7 @@ README of all Packages
    bob.bio.spear <bob/bob.bio.spear/README.rst>
    bob.bio.vein <bob/bob.bio.vein/README.rst>
    bob.bio.video <bob/bob.bio.video/README.rst>
+   bob.devtools <bob/bob.devtools/README.rst>
    bob.extension <bob/bob.extension/README.rst>
    bob.fusion.base <bob/bob.fusion.base/README.rst>
    bob.io.base <bob/bob.io.base/README.rst>

doc/tutorial.rst

+.. _bob.tutorial:
+
+********************************
+ Getting started with |project|
+********************************
+
+The following tutorial constitutes a suitable starting point to get to
+know how to use |project|'s packages and to learn its fundamental concepts.
+
+
+Multi-dimensional Arrays
+========================
+
+The fundamental data structure of |project| is a multi-dimensional array. In
+signal processing and machine learning, arrays are a suitable representation
+for many different types of digital signals such as images, audio data and
+extracted features. For multi-dimensional arrays, we rely on `NumPy`_.
+For an introduction and tutorials about NumPy ndarrays, just visit the `NumPy
+Reference`_ website.
+
+
+Digital signals as multi-dimensional arrays
+===========================================
+
+For |project|, we have decided to represent digital signals directly as
+:any:`numpy.ndarray` rather than having dedicated classes for each type of
+signals. This implies that some convention has been defined.
+
+Vectors and matrices
+--------------------
+
+A vector is represented as a 1D NumPy array, whereas a matrix is
+represented by a 2D array whose first dimension corresponds to the rows,
+and second dimension to the columns.
+
+.. code:: python
+
+    >>> import numpy
+    >>> A = numpy.array([[1, 2, 3], [4, 5, 6]], dtype='uint8') # A is a matrix 2x3
+    >>> print(A)
+    [[1 2 3]
+     [4 5 6]]
+    >>> b = numpy.array([1, 2, 3], dtype='uint8') # b is a vector of length 3
+    >>> print(b)
+    [1 2 3]
+
+Images
+------
+
+**Grayscale** images are represented as 2D arrays, the first dimension
+being the height (number of rows) and the second dimension being the
+width (number of columns). For instance:
+
+.. code:: python
+
+    >>> img = numpy.ndarray((480,640), dtype='uint8')
+
+``img`` which is a 2D array can be seen as a gray-scale image of
+dimension 640 (width) by 480 (height). In addition, ``img`` can be seen
+as a matrix with 480 rows and 640 columns. This is the reason why we
+have decided that for images, the first dimension is the height and the
+second one the width, such that it matches the matrix convention as
+well.
+
+**Color** images are represented as 3D arrays, the first dimension being
+the number of color planes, the second dimension the height and the
+third the width. As an image is an array, this is the responsibility of
+the user to know in which color space the content is stored.
+:any:`bob.io.image` provides functions to convert Bob format images into
+Matplotlib_ and other formats and back:
+
+.. code:: python
+
+    >>> import bob.io.image
+    >>> colored_bob_format = numpy.ndarray((3,480,640), dtype='uint8')
+    >>> colored_matplotlib_format = bob.io.image.to_matplotlib(colored_bob_format)
+    >>> print(colored_matplotlib_format.shape)
+    [480 640 3]
+    >>> colored_bob_format = bob.io.image.to_bob(colored_matplotlib_format)
+    >>> print(colored_bob_format.shape)
+    [3 480 640]
+    >>> pillow_img = bob.io.image.bob_to_pillow(colored_bob_format)
+    >>> opencv_bgr = bob.io.image.bob_to_opencv(colored_bob_format)
+
+.. note::
+
+    In :ref:`bob.bio.face`, the images are assumed to be in range ``[0,255]``
+    irrespective of their data type.
+
+Videos
+------
+
+A video can be seen as a sequence of images over time. By convention, the first
+dimension is for the frame indices (time index), whereas the remaining ones are
+related to the corresponding image frame. The videos have the shape of
+``(N,C,H,W)``, where ``N`` is the number of frames, ``H`` the height, ``W`` the
+width and ``C`` the number of color planes.
+
+
+Input and output
+================
+
+:ref:`bob.io.base` provides two generic functions :any:`bob.io.base.load` and
+:any:`bob.io.base.save` to load and save data of various types, based on the
+filename extension. For example, to load a ``.jpg`` image, simply call:
+
+.. code:: python
+
+    >>> import bob.io.base
+    >>> img = bob.io.base.load("myimg.jpg")
+
+`HDF5`_ format, through h5py_, and images, through imageio_, are supported.
+For loading videos, use imageio-ffmpeg_ directly.
+
+
+Machine learning
+================
+
+:ref:`bob.learn.em` provides implementation of the following methods:
+
+    - K-Means clustering
+    - Gaussian Mixture Modeling (GMM)
+    - Joint Factor Analysis (JFA)
+    - Inter-Session Variability (ISV)
+    - Total Variability (TV, also known as i-vector)
+    - Probabilistic Linear Discriminant Analysis (PLDA, also known as i-vector)
+
+All implementations use dask_ to parallelize the training computation.
+
+Database interfaces
+===================
+
+Bob provides an API on top of CSV files to easily query databases.
+A generic implementation is provided in :ref:`bob.pipelines` but packages
+such as :ref:`bob.bio.base` and :ref:`bob.pad.base` provide their own implementations.
+
+Performance evaluation
+======================
+
+Methods in the :ref:`bob.measure` module can be used evaluate error for
+multi-class or binary classification problems. Several evaluation
+techniques such as:
+
+    - Root Mean Squared Error (RMSE)
+    - F-score
+    - Precision and Recall
+    - False Positive Rate (FPR)
+    - False Negative Rate (FNR)
+    - Equal Error Rates (EER)
+
+can be computed. Moreover, functionality for plotting
+
+    - ROC
+    - DET
+    - CMC
+    - EPC
+
+curves are described in more detail in the :ref:`bob.measure`.
+
+.. include:: links.rst