Commit d131a79e authored by Yannick DAYER's avatar Yannick DAYER

[doc] Change spoof.py to bob pad

parent af2ae899
......@@ -7,21 +7,21 @@
Executing Baseline Algorithms
===============================
This section explains how to execute face presentation attack detection (PAD) algorithms implemented
in ``bob.pad.face``.
This section explains how to execute face presentation attack detection (PAD)
algorithms implemented in ``bob.pad.face``.
Running Baseline Experiments
----------------------------
To run the baseline PAD experiments, the ``bob pad vanilla-pad`` script is used.
To see the description of the script you can type in the console:
To run the baseline PAD experiments, the ``bob pad vanilla-pad`` command is used.
To see the description of the command, you can type in the console:
.. code-block:: sh
$ bob pad vanilla-pad --help
This script is explained in more detail in :ref:`bob.pad.base.experiments`.
This script is explained in more detail in :ref:`bob.pad.base <bob.pad.base.vanilla_pad_intro>`.
Usually it is a good idea to have at least verbose level 2 (i.e., calling
``bob pad vanilla-pad --verbose --verbose``, or the short version
......@@ -30,17 +30,17 @@ Usually it is a good idea to have at least verbose level 2 (i.e., calling
.. note:: **Running in Parallel**
To run the experiments in parallel, you can define a dask client
(multi-processing) configurations as explained in
:ref:`running_in_parallel`.
(multi-processing) configurations as explained in :ref:`running_in_parallel`.
In short, to run in the Idiap SGE grid, you can simply add the ``--grid``
command line option, with grid configuration parameters. To run experiments in parallel on
the local machine, simply add a ``--parallel <N>`` option, where ``<N>``
specifies the number of parallel jobs you want to execute.
In short, to run in the Idiap SGE grid, you can simply add the ``-l``
(``--dask-client``) command line option, with grid configuration parameters
(e.g. ``-l sge``).
To run experiments in parallel on the local machine, simply add the
``-l local-parallel`` option.
Database setups and baselines are encoded using
``bob.bio.base.configuration-files``, all stored inside the package root, in
``configuration-files``, all stored inside the package structure, in
the directory ``bob/pad/face/config``. Documentation for each resource
is available on the section :ref:`bob.pad.face.resources`.
......@@ -50,29 +50,26 @@ is available on the section :ref:`bob.pad.face.resources`.
instructions described in this guide. You **need first** to procure yourself
the raw data files that correspond to *each* database used here in order to
correctly run experiments with those data. Biometric data is considered
private date and, under EU regulations, cannot be distributed without a
private data and, under EU regulations, cannot be distributed without a
consent or license. You may consult our
:ref:`bob.pad.face.resources.databases` resources section for checking
currently supported databases and accessing download links for the raw data
files.
Once the raw data files have been downloaded, particular attention should be
given to the directory locations of those. Unpack the databases carefully
and annotate the root directory where they have been unpacked.
Once the raw data files have been downloaded, unpack the databases carefully
and take a note of the root directory where they have been unpacked.
Then, carefully read the *Databases* section of
:ref:`bob.pad.base.installation` on how to correctly setup the
``~/.bob_bio_databases.txt`` file.
Use the following commands to specify the correct parameters of your dataset
(see :ref:`bob.pad.face.resources.databases`):
Use the following keywords on the left side of the assignment (see
:ref:`bob.pad.face.resources.databases`):
.. code-block:: sh
.. code-block:: text
$ bob config set bob.db.replaymobile.directory /path/to/replayattack-database/
$ bob config set bob.db.replaymobile.extension .mov
[YOUR_REPLAY_ATTACK_DIRECTORY] = /complete/path/to/replayattack-database/
Notice it is rather important to use the strings as described above,
otherwise ``bob.pad.base`` will not be able to correctly load your images.
Notice it is rather important to correctly configure the database as
described above, otherwise ``bob.pad.base`` will not be able to correctly
load your data.
Once this step is done, you can proceed with the instructions below.
......@@ -84,31 +81,39 @@ is available on the section :ref:`bob.pad.face.resources`.
Baselines on REPLAY-ATTACK database
--------------------------------------
This section summarizes the results of baseline face PAD experiments on the REPLAY-ATTACK (`replayattack`_) database.
The description of the database-related settings, which are used to run face PAD baselines on the Replay-Attack is given here :ref:`bob.pad.face.resources.databases.replay`. To understand the settings in more details you can check the corresponding configuration file : ``bob/pad/face/config/replay_attack.py``.
This section summarizes the results of baseline face PAD experiments on the
REPLAY-ATTACK (`replayattack`_) database.
The description of the database-related settings, which are used to run face PAD
baselines on the Replay-Attack is given here
:ref:`bob.pad.face.resources.databases.replay`. To understand the settings in
more details you can check the corresponding configuration file :
``bob/pad/face/config/replay_attack.py``.
LBP features of facial region + SVM classifier
===================================================
Detailed description of this PAD pipe-line is given at :ref:`bob.pad.face.resources.face_pad.lbp_svm_replayattack`.
Detailed description of this PAD pipe-line is given at
:ref:`bob.pad.face.resources.face_pad.lbp_svm_replayattack`.
To run this baseline on the `replayattack`_ database, using the ``grandtest`` protocol, execute the following:
To run this baseline on the `replayattack`_ database, using the ``grandtest``
protocol, execute the following:
.. code-block:: sh
$ bob pad vanilla-pad replay-attack lbp-64 svm-frames
$ bob pad vanilla-pad replay-attack lbp svm-frames -o <PATH_TO_STORE_THE_RESULTS>
.. tip::
If you are in `idiap`_ you can use SGE grid to speed-up the calculations.
Simply add ``--grid idiap`` argument to the above command. For example:
If you are at `idiap`_ you can use SGE grid to speed-up the calculations.
Simply add the ``--dask-client sge`` (or ``-l sge``) argument to the above
command. For example:
.. code-block:: sh
$ spoof.py replay-attack lbp-svm \
--sub-directory <PATH_TO_STORE_THE_RESULTS> \
--grid idiap
$ bob pad vanilla-pad replay-attack lbp svm-frames \
--output <PATH_TO_STORE_THE_RESULTS> \
--dask-client idiap
To understand the settings of this baseline PAD experiment you can check the
corresponding configuration file: ``bob/pad/face/config/lbp_svm.py``
......@@ -119,15 +124,15 @@ following command:
.. code-block:: sh
bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
<PATH_TO_STORE_THE_RESULTS>/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/scores-eval \
--legends "LBP features of facial region + SVM classifier + REPLAY-ATTACK database" \
-e \
--criterion eer \
-o <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
The EER/HTER errors for `replayattack`_ database are summarized in the Table below:
The error rates for `replayattack`_ database are summarized in the Table below:
+-------------------+----------+----------+
| Protocol | EER,\% | HTER,\% |
......@@ -135,7 +140,7 @@ The EER/HTER errors for `replayattack`_ database are summarized in the Table bel
| ``grandtest`` | 15.117 | 15.609 |
+-------------------+----------+----------+
The ROC curves for the particular experiment can be downloaded from here:
The ROC curves for this particular experiment can be downloaded from here:
:download:`ROC curve <img/ROC_lbp_svm_replay_attack.pdf>`
......@@ -145,13 +150,13 @@ The ROC curves for the particular experiment can be downloaded from here:
Image Quality Measures as features of facial region + SVM classifier
========================================================================
Detailed description of this PAD pipe-line is given at :ref:`bob.pad.face.resources.face_pad.qm_svm_replayattack`.
Detailed description of this PAD pipeline is given at :ref:`bob.pad.face.resources.face_pad.qm_svm_replayattack`.
To run this baseline on the `replayattack`_ database, using the ``grandtest`` protocol, execute the following:
.. code-block:: sh
$ spoof.py replay-attack qm-svm \
$ bob pad vanilla-pad replay-attack qm svm-frames \
--sub-directory <PATH_TO_STORE_THE_RESULTS>
.. tip::
......@@ -209,8 +214,8 @@ To run this baseline on the `Replay-Mobile`_ database, using the ``grandtest`` p
.. code-block:: sh
$ spoof.py replay-mobile lbp-svm \
--sub-directory <PATH_TO_STORE_THE_RESULTS>
$ bob pad vanilla-pad replay-mobile lbp svm_frame \
--output <PATH_TO_STORE_THE_RESULTS>
.. tip::
......@@ -257,8 +262,8 @@ To run this baseline on the `Replay-Mobile`_ database, using the ``grandtest`` p
.. code-block:: sh
$ spoof.py replay-mobile qm-svm \
--sub-directory <PATH_TO_STORE_THE_RESULTS>
$ bob pad vanilla-pad replay-mobile qm svm-frames \
--output <PATH_TO_STORE_THE_RESULTS>
.. tip::
......@@ -314,8 +319,8 @@ To run this baseline on the MIFS database, using the ``grandtest`` protocol, exe
.. code-block:: sh
$ spoof.py mifs lbp-svm \
--sub-directory <PATH_TO_STORE_THE_RESULTS>
$ bob pad vanilla-pad mifs lbp svm-frames \
--output <PATH_TO_STORE_THE_RESULTS>
To evaluate the results computing EER, HTER and plotting ROC you can use the
following command:
......@@ -348,8 +353,8 @@ To run this baseline on the MIFS database, using the ``grandtest`` protocol, exe
.. code-block:: sh
$ spoof.py mifs qm-svm \
--sub-directory <PATH_TO_STORE_THE_RESULTS>
$ bob pad vanilla-pad mifs qm svm-frames \
--output <PATH_TO_STORE_THE_RESULTS>
To evaluate the results computing EER, HTER and plotting ROC you can use the
following command:
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment