Skip to content
Snippets Groups Projects
Commit 09a9eaf4 authored by Olegs NIKISINS's avatar Olegs NIKISINS
Browse files

Added the doc on how to run the baseline PAD system with 1 MC AE

parent 5ad94489
No related branches found
No related tags found
No related merge requests found
Pipeline #32965 failed with stages
in 13 minutes and 3 seconds
.. _bob.paper.mcae.icb2019.1_mc_ae_face_pad:
===============================================================
Multi-channel face PAD using 1 autoencoder (baseline system)
===============================================================
This section explains how to run a complete face PAD experiment using multi-channel autoencoder-based face PAD system, as well as a training work-flow.
The system discussed in this section is introduced the following publication [NGM19]_. It is **strongly recommended** to check the publication for better understanding
of the described work-flow.
.. note::
Below discussions rely on the fact, that two databases, CelebA and WMCA, are downloaded and installed in your system. Please refer to :ref:`bob.pad.face.baselines` section in the documentation of ``bob.pad.face`` package for more details on how to run the face PAD experiments and setup the databases.
Training multi-channel autoencoder-based face PAD system.
----------------------------------------------------------------
The training procedure of multi-channel face PAD system using **one** autoencoder is thoroughly discussed in the documentation of ``bob.pad.face`` package, section on :ref:`bob.pad.face.mc_autoencoder_pad`, thus it is not repeated in current instructions.
Running face PAD Experiments
------------------------------
After the training procedure is completed, one can run a complete PAD experiment. The command to run **two** PAD experiments on the multi-channel, BW-NIR-D, data of WMCA:
.. code-block:: sh
./bin/spoof.py \ # spoof.py is used to run the experiment
batl-db-rgb-ir-d-grandtest \ # WMCA database instance allowing to loading RGB-NIR-D channels
mc-pad-bw-nir-d-1-ae-celeba-pretrain \ # configuration defining Preprocessor, Extractor, and Algorithm instances
--allow-missing-files \ # don't stop the execution if some files are missing
--grid idiap \ # use grid, only for Idiap users, REMOVE otherwise
--sub-directory <FOLDER_TO_SAVE_THE_RESULTS_1> # define your path here
./bin/spoof.py \
batl-db-rgb-ir-d-grandtest \
mc-pad-bw-nir-d-1-ae-batl-3-layers-finetune \
--allow-missing-files \
--grid idiap \
--sub-directory <FOLDER_TO_SAVE_THE_RESULTS_2>
The **first** experiment corresponds to the PAD system composed of: preprocessor extracting 1 MC facial patch (entire face), extractor with 1 autoencoder trained on CelebA only, and an MLP algorithm trained on MC latent embeddings.
The **second** experiment corresponds to the PAD system composed of: preprocessor extracting 1 MC facial patch (entire face), extractor with 1 autoencoder trained on CelebA and fine-tuned on the WMCA, and an MLP algorithm trained on MC latent embeddings.
.. note::
Entry points ``mc-pad-bw-nir-d-1-ae-celeba-pretrain`` and ``mc-pad-bw-nir-d-1-ae-batl-3-layers-finetune`` are pointing to corresponding configuration files, which initialize autoencoders and MLP using pre-trained models distributed with the package. If you want to use the models computed in the previous steps, revise the configuration files pointing to models from your system.
After an execution of above scripts is completed the score files will be located in the folders: ``<FOLDER_TO_SAVE_THE_RESULTS_1>/grandtest-color*infrared*depth-10/scores/`` and ``<FOLDER_TO_SAVE_THE_RESULTS_2>/grandtest-color*infrared*depth-10/scores/``.
Evaluating results of face PAD Experiments
---------------------------------------------
Assuming the score files, for **two** experiments discussed above, are located in the folders ``<FOLDER_TO_SAVE_THE_RESULTS_1>/grandtest-color*infrared*depth-10/scores/`` and ``<FOLDER_TO_SAVE_THE_RESULTS_2>/grandtest-color*infrared*depth-10/scores/``, one can use the following command to evaluate the results:
.. code-block:: sh
./bin/bob bio evaluate -e -v -l '<SAVE_FOLDER>/metrix.txt' \
-o '<SAVE_FOLDER>/curves.pdf' \
-lg "1 MC patch + CelebA AE + 10 ReLU",\
"1 MC patch + CelebA & BATL 3-layer-tune AE + 10 ReLU" \
<FOLDER_TO_SAVE_THE_RESULTS_1>/grandtest-color*infrared*depth-10/scores/scores-dev \
<FOLDER_TO_SAVE_THE_RESULTS_1>/grandtest-color*infrared*depth-10/scores/scores-eval \
<FOLDER_TO_SAVE_THE_RESULTS_2>/grandtest-color*infrared*depth-10/scores/scores-dev \
<FOLDER_TO_SAVE_THE_RESULTS_2>/grandtest-color*infrared*depth-10/scores/scores-eval
The files **metrix.txt** and **curves.pdf**, containing error rates and evaluation curves, are saved to ``<SAVE_FOLDER>`` location. Inspecting the **curves.pdf**, one can find the curves similar to this one:
.. figure:: img/roc_1_ae_pad_system.png
:align: center
ROC curves for PAD system using 1 AE for MC face, and an MLP classifier. For **evaluation** set of WMCA, **grandtest-10** protocol.
.. include:: links.rst
......@@ -3,9 +3,9 @@
.. _bob.paper.mcae.icb2019.9_mc_ae_face_pad:
=============================================
Multi-channel face PAD using 9 autoencoders
=============================================
============================================================
Multi-channel face PAD using 9 autoencoders (best system)
============================================================
This section explains how to run a complete face PAD experiment using multi-channel autoencoder-based face PAD system, as well as a training work-flow.
......@@ -605,7 +605,7 @@ After the training procedure, discussed above, is completed, one can run a compl
mc-pad-bw-nir-d-9-ae-batl-3-layers-finetune \
--allow-missing-files \
--grid idiap \
--sub-directory <FOLDER_TO_SAVE_THE_RESULTS_1>
--sub-directory <FOLDER_TO_SAVE_THE_RESULTS_2>
The **first** experiment corresponds to the PAD system composed of: preprocessor extracting 9 MC facial patches, extractor with 9 autoencoders trained on CelebA only, and an MLP algorithm trained on MC latent embeddings.
......
doc/img/roc_1_ae_pad_system.png

59.9 KiB

......@@ -26,6 +26,7 @@ User guide
:maxdepth: 2
9_mc_ae_face_pad
1_mc_ae_face_pad
references
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment