Now everything is ready to run the experiments. Here is a generic command for training GMM-based PAD system::
By using ``~/.bob_bio_databases.txt`` file, where path placeholders are replaced with real paths, bob's framework
packages can find the data needed for the experiments.
GMM models for ``real`` and ``spoofed`` data are trained as per this generic command::
$ ./bin/train_gmm.py -d DB_NAME -p Preprocessor -e Feature_Extractor -a Classifier -s Folder_Name --groups world --skip-enroller-training -vv --parallel 6
$ ./bin/train_gmm.py -d DB_NAME -p Preprocessor -e Feature_Extractor -a gmm-tomi -s Folder_Name --groups world --skip-enroller-training -vv
This training may take a long time and as the results, it will generate GMM model and write it into ``Projector.hdf5``
file. You can check all possible options by running ``$ ./bin/train_gmm.py --help``.
Here is the generic command to tune the system on developing set and evaluate on the test set::
Here is the generic command for tuning the trained system on developing set and evaluate on the test set::
$ ./bin/spoof.py -d DB_NAME -p Preprocessor -e Feature_Extractor -a gmm --projector-file Projector_spoof.hdf5 -s Folder_Name --groups dev eval --skip-projector-training -vv
$ ./bin/spoof.py -d DB_NAME -p Preprocessor -e Feature_Extractor -a Classifier --projector-file Projector_spoof.hdf5 -s Folder_Name --groups dev eval --skip-projector-training -vv
For example, to train and evaluate a GMM-based PAD system using MFCC-based features for Licit protocol of the ASVspoof database, the following commands need to be run::
For example, to train and evaluate a GMM-based PAD system using MFCC-based features computed for
``licit`` and ``spoof`` protocols of the ASVspoof database, the following commands need to be run::
$ ./bin/train_gmm.py -d asvspoof-licit -p mod-4hz -e mfcc20 -a gmm-tomi -s temp --groups world
--skip-enroller-training -vv --parallel 6
$ ./bin/train_gmm.py -d asvspoof-spoof -p mod-4hz -e mfcc20 -a gmm-tomi -s temp --groups world
Also, it is possible to reproduce the experiments presented in the paper using the following bash scripts that run for all PAD systems used::
Also, it is possible to reproduce the experiments presented in the paper using the following bash scripts that run for all
PAD systems used in the paper (note that these scripts assume SGE grid to be available and will take a few days on 50
parallel machines)::
$ ./train_gmms.sh avspoof 20 # train for AVspoof database
$ ./train_gmms.sh asvspoof 20 # train for ASVspoof database
...
...
@@ -80,7 +108,8 @@ If you want to avoid training all PAD systems and computing scores, we are provi
$ tar -xzvf interspeech_2016.tar.gz
$ ./evaluate_scores.sh # compute error rates and plot the DET curves for each PAD system
The script will create folders for each different PAD system (it contains computed error rates and DET curves) and one ``stats.txt`` file with error rates from all systems in one LaTeX table.
The script will create folders for each different PAD system (it contains computed error rates and DET curves)
and one ``stats.txt`` file with error rates from all systems in one LaTeX table.
To plot combined DET curves for different systems as per Figure 2 of the paper, the following script can be run::
...
...
@@ -88,13 +117,8 @@ To plot combined DET curves for different systems as per Figure 2 of the paper,
This script will plot several DET curves in a single PDF file inside the folder ``plots_compare_pads``.
Installation
------------
To install this package -- alone or together with other `Packages of Bob <https://github.com/idiap/bob/wiki/Packages>`_ -- please read the `Installation Instructions <https://github.com/idiap/bob/wiki/Installation>`_.
For Bob_ to be able to work properly, some dependent packages are required to be installed.
Please make sure that you have read the `Dependencies <https://github.com/idiap/bob/wiki/Dependencies>`_ for your operating system.
