baselines.rst

Executing Baseline Algorithms

This section explains how to execute face presentation attack detection (PAD) algorithms implemented in bob.pad.face.

Running Baseline Experiments

To run the baseline PAD experiments, the spoof.py script located in bin directory is used. To see the description of the script you can type in the console:

$ ./bin/verify.py --help

This script is explained in more detail in :ref:`bob.pad.base.experiments`.

Usually it is a good idea to have at least verbose level 2 (i.e., calling spoof.py --verbose --verbose, or the short version spoof.py -vv).

Note

Running in Parallel

To run the experiments in parallel, you can define an SGE grid or local host (multi-processing) configurations as explained in :ref:`running_in_parallel`.

In short, to run in the Idiap SGE grid, you can simply add the --grid command line option, with grid configuration parameters. To run experiments in parallel on the local machine, simply add a --parallel <N> option, where <N> specifies the number of parallel jobs you want to execute.

Database setups and baselines are encoded using :ref:`bob.bio.base.configuration-files`, all stored inside the package root, in the directory bob/pad/face/config. Documentation for each resource is available on the section :ref:`bob.pad.face.resources`.

Warning

You cannot run experiments just by executing the command line instructions described in this guide. You need first to procure yourself the raw data files that correspond to each database used here in order to correctly run experiments with those data. Biometric data is considered private date and, under EU regulations, cannot be distributed without a consent or license. You may consult our :ref:`bob.pad.face.resources.databases` resources section for checking currently supported databases and accessing download links for the raw data files.

Once the raw data files have been downloaded, particular attention should be given to the directory locations of those. Unpack the databases carefully and annotate the root directory where they have been unpacked.

Then, carefully read the Databases section of :ref:`bob.pad.base.installation` on how to correctly setup the ~/.bob_bio_databases.txt file.

Use the following keywords on the left side of the assignment (see :ref:`bob.pad.face.resources.databases`):

[YOUR_REPLAY_ATTACK_DIRECTORY] = /complete/path/to/replayattack-database/

Notice it is rather important to use the strings as described above, otherwise bob.pad.base will not be able to correctly load your images.

Once this step is done, you can proceed with the instructions below.

---

Baselines on REPLAY-ATTACK database

This section summarizes the results of baseline face PAD experiments on the REPLAY-ATTACK (`replayattack`_) database.

LBP features of facial region + SVM classifier

Detailed description of this PAD pipe-line is given at :ref:`bob.pad.face.resources.face_pad.lbp_svm_replayattack`.

To run this baseline on the `replayattack`_ database, using the grandtest protocol, execute the following:

$ ./bin/spoof.py lbp-svm \
--database replay --protocol grandtest --groups train dev eval \
--sub-directory <PATH_TO_STORE_THE_RESULTS>

Tip

If you are in `idiap`_ you can use SGE grid to speed-up the calculations. Simply add --grid idiap argument to the above command. For example:

$ ./bin/spoof.py lbp-svm \
--database replay --protocol grandtest --groups train dev eval \
--sub-directory <PATH_TO_STORE_THE_RESULTS> \
--grid idiap

To understand the settings of this baseline PAD experiment you can check the corresponding configuration file: bob/pad/face/config/lbp_svm.py

To evaluate the results computing EER, HTER and plotting ROC you can use the following command:

./bin/evaluate.py \
--dev-files  <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev  \
--eval-files <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
--legends "LBP features of facial region + SVM classifier + REPLAY-ATTACK database" \
-F 7 \
--criterion EER \
--roc <PATH_TO_STORE_THE_RESULTS>/ROC.pdf

The EER/HTER errors for `replayattack`_ database are summarized in the Table below:

Protocol	EER,%	HTER,%
grandtest	15.117	15.609

The ROC curves for the particular experiment can be downloaded from here:

:download:`ROC curve <img/ROC_lbp_svm_replay_attack.pdf>`

---

Image Quality Measures as features of facial region + SVM classifier

Detailed description of this PAD pipe-line is given at :ref:`bob.pad.face.resources.face_pad.qm_svm_replayattack`.

To run this baseline on the `replayattack`_ database, using the grandtest protocol, execute the following:

$ ./bin/spoof.py qm-svm \
--database replay --protocol grandtest --groups train dev eval \
--sub-directory <PATH_TO_STORE_THE_RESULTS>

Tip

Similarly to the tip above you can run this baseline in parallel.

To understand the settings of this baseline PAD experiment you can check the corresponding configuration file: bob/pad/face/config/qm_svm.py

To evaluate the results computing EER, HTER and plotting ROC you can use the following command:

./bin/evaluate.py \
--dev-files  <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev  \
--eval-files <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
--legends "IQM features of facial region + SVM classifier + REPLAY-ATTACK database" \
-F 7 \
--criterion EER \
--roc <PATH_TO_STORE_THE_RESULTS>/ROC.pdf

The EER/HTER errors for `replayattack`_ database are summarized in the Table below:

Protocol	EER,%	HTER,%
grandtest	4.321	4.570

The ROC curves for the particular experiment can be downloaded from here:

:download:`ROC curve <img/ROC_iqm_svm_replay_attack.pdf>`

---