Skip to content
Snippets Groups Projects
Commit 44eb214b authored by Amir MOHAMMADI's avatar Amir MOHAMMADI
Browse files

Merge branch 'fix-doc' into 'master' 

[doc] Update the baselines

See merge request !118
parents 4e36338d a01d56d4
No related branches found
No related tags found
1 merge request!118[doc] Update the baselines
Pipeline #52096 passed
Stage: build
Stage: deploy
Hide whitespace changes
Inline Side-by-side
Showing
with 130 additions and 145 deletions
bob/pad/face/database/database.py
+ 1
− 1
View file @ 44eb214b
......@@ -145,7 +145,7 @@ class VideoPadFile(PadFile):
Returns
-------
collection.Iterator
collections.abc.Iterator
An iterator returning frames of the video.
"""
path = self.make_path(
......
bob/pad/face/extractor/LBPHistogram.py
+ 3
− 4
View file @ 44eb214b
......@@ -19,8 +19,7 @@ class LBPHistogram(TransformerMixin, BaseEstimator):
to_average : bool
Compare the neighbors to the average of the pixels instead of the central pixel?
radius : float
The radius of the circle on which the points are taken (for circular
LBP)
The radius of the circle on which the points are taken (for circular LBP)
neighbors : int
The number of points around the central point on which LBP is
computed (4, 8, 16)
......@@ -36,9 +35,9 @@ class LBPHistogram(TransformerMixin, BaseEstimator):
Attributes
----------
dtype : numpy.dtype
If a ``dtype`` is specified in the contructor, it is assured that the
If a ``dtype`` is specified in the constructor, it is assured that the
resulting features have that dtype.
lbp : LBP
lbp : bob.ip.base.LBP
The LPB extractor object.
"""
......
bob/pad/face/utils/load_utils.py
+ 12
− 12
View file @ 44eb214b
......@@ -20,7 +20,7 @@ def frames(path):
Yields
------
:any:`numpy.array`
numpy.ndarray
A frame of the video. The size is (3, 240, 320).
"""
video = reader(path)
......@@ -69,16 +69,16 @@ def yield_faces(pad_sample, cropper, normalizer=None):
----------
pad_sample
The pad sample to return the faces.
cropper : callable
cropper : collections.abc.Callable
A face image cropper that works with database's annotations.
normalizer : callable
normalizer : collections.abc.Callable
If not None, it should be a function that takes all the annotations of
the whole video and yields normalized annotations frame by frame. It
should yield same as ``annotations.items()``.
Yields
------
numpy.array
numpy.ndarray
Face images
Raises
......@@ -110,7 +110,7 @@ def scale_face(face, face_height, face_width=None):
Parameters
----------
face : :any:`numpy.array`
face : numpy.ndarray
The face image. It can be 2D or 3D in bob image format.
face_height : int
The height of the scaled face.
......@@ -119,7 +119,7 @@ def scale_face(face, face_height, face_width=None):
Returns
-------
:any:`numpy.array`
numpy.ndarray
The scaled face.
"""
face_width = face_height if face_width is None else face_width
......@@ -135,7 +135,7 @@ def blocks(data, block_size, block_overlap=(0, 0)):
Parameters
----------
data : :any:`numpy.array`
data : numpy.ndarray
The image in gray-scale, color, or color video format.
block_size : (int, int)
The size of patches
......@@ -144,7 +144,7 @@ def blocks(data, block_size, block_overlap=(0, 0)):
Returns
-------
:any:`numpy.array`
numpy.ndarray
The patches.
Raises
......@@ -180,7 +180,7 @@ def blocks_generator(data, block_size, block_overlap=(0, 0)):
Parameters
----------
data : :any:`numpy.array`
data : numpy.ndarray
The image in gray-scale, color, or color video format.
block_size : (int, int)
The size of patches
......@@ -189,7 +189,7 @@ def blocks_generator(data, block_size, block_overlap=(0, 0)):
Yields
------
:any:`numpy.array`
numpy.ndarray
The patches.
Raises
......@@ -215,7 +215,7 @@ def color_augmentation(image, channels=("rgb",)):
Parameters
----------
image : numpy.array
image : numpy.ndarray
The image in RGB Bob format.
channels : :obj:`tuple`, optional
List of channels to convert the image to. It can be any of ``rgb``,
......@@ -223,7 +223,7 @@ def color_augmentation(image, channels=("rgb",)):
Returns
-------
numpy.array
numpy.ndarray
The image that contains several channels:
``(3*len(channels), height, width)``.
"""
......
doc/api.rst
+ 2
− 2
View file @ 44eb214b
......@@ -21,12 +21,12 @@ Base classes
REPLAY-ATTACK Database
========================
.. autoclass:: bob.pad.face.database.replay_attack.ReplayAttackPadDatabase
.. autofunction:: bob.pad.face.database.replay_attack.ReplayAttackPadDatabase
REPLAY-MOBILE Database
========================
.. autoclass:: bob.pad.face.database.replay_mobile.ReplayMobilePadDatabase
.. autofunction:: bob.pad.face.database.replay_mobile.ReplayMobilePadDatabase
......
doc/baselines.rst
+ 112
− 126
View file @ 44eb214b
......@@ -100,7 +100,8 @@ protocol, execute the following:
.. code-block:: sh
$ bob pad vanilla-pad replay-attack lbp svm-frames -o <PATH_TO_STORE_THE_RESULTS>
$ bob pad vanilla-pad replay-attack lbp svm-frames \
--output <PATH_TO_STORE_THE_RESULTS>
.. tip::
......@@ -112,7 +113,7 @@ protocol, execute the following:
$ bob pad vanilla-pad replay-attack lbp svm-frames \
--output <PATH_TO_STORE_THE_RESULTS> \
--dask-client idiap
--dask-client sge
To understand the settings of this baseline PAD experiment you can check the
corresponding configuration file: ``bob/pad/face/config/lbp_svm.py``
......@@ -122,22 +123,37 @@ following command:
.. code-block:: sh
bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/scores-eval \
$ bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/scores-dev.csv \
<PATH_TO_STORE_THE_RESULTS>/scores-eval.csv \
--legends "LBP features of facial region + SVM classifier + REPLAY-ATTACK database" \
-e \
--eval \
--criterion eer \
-o <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
--output <PATH_TO_STORE_THE_RESULTS>/evaluation_report.pdf
The error rates for `replayattack`_ database are summarized in the table below:
+-------------------+----------+----------+
| Protocol | EER,\% | HTER,\% |
+===================+==========+==========+
| ``grandtest`` | 15.117 | 15.609 |
+-------------------+----------+----------+
============== ================= =================
.. Development Evaluation
============== ================= =================
APCER (attack) 17.4% 14.2%
APCER_AP 17.4% 14.2%
BPCER 17.4% 16.4%
ACER 17.4% 15.3%
FTA 0.0% 0.0%
FPR 17.4% (1045/6000) 14.2% (1134/7999)
FNR 17.4% (209/1200) 16.4% (262/1600)
HTER 17.4% 15.3%
FAR 17.4% 14.2%
FRR 17.4% 16.4%
PRECISION 0.5 0.5
RECALL 0.8 0.8
F1_SCORE 0.6 0.7
AUC 0.9 0.9
AUC-LOG-SCALE 2.0 2.1
============== ================= =================
The ROC curves for this particular experiment can be downloaded from here:
......@@ -156,11 +172,12 @@ To run this baseline on the `replayattack`_ database, using the ``grandtest`` pr
.. code-block:: sh
$ bob pad vanilla-pad replay-attack qm svm-frames \
--sub-directory <PATH_TO_STORE_THE_RESULTS>
--output <PATH_TO_STORE_THE_RESULTS>
.. tip::
Similarly to the tip above you can run this baseline in parallel.
Similarly to the tip above you can run this baseline in parallel with the
``--dask-client`` option.
To understand the settings of this baseline PAD experiment you can check the
corresponding configuration file: ``bob/pad/face/config/qm_svm.py``
......@@ -170,21 +187,36 @@ following command:
.. code-block:: sh
bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
$ bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/scores-dev.csv \
<PATH_TO_STORE_THE_RESULTS>/scores-eval.csv \
--legends "IQM features of facial region + SVM classifier + REPLAY-ATTACK database" \
-e \
--eval \
--criterion eer \
-o <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
--output <PATH_TO_STORE_THE_RESULTS>/evaluation_report.pdf
The EER/HTER errors for `replayattack`_ database are summarized in the table below:
+-------------------+----------+----------+
| Protocol | EER,\% | HTER,\% |
+===================+==========+==========+
| ``grandtest`` | 3.987 | 4.571 |
+-------------------+----------+----------+
============== ================ ==================
.. Development Evaluation
============== ================ ==================
APCER (attack) 6.58% 12.96%
APCER_AP 6.58% 12.96%
BPCER 6.58% 0.19%
ACER 6.58% 6.58%
FTA 0.00% 0.00%
FPR 6.58% (395/6000) 12.96% (1037/7999)
FNR 6.58% (79/1200) 0.19% (3/1600)
HTER 6.58% 6.58%
FAR 6.58% 12.96%
FRR 6.58% 0.19%
PRECISION 0.74 0.61
RECALL 0.93 1.00
F1_SCORE 0.83 0.75
AUC 0.98 0.99
AUC-LOG-SCALE 2.91 2.71
============== ================ ==================
The ROC curves for the particular experiment can be downloaded from here:
......@@ -218,7 +250,8 @@ To run this baseline on the `Replay-Mobile`_ database, using the ``grandtest`` p
.. tip::
Similarly to the tip above you can run this baseline in parallel.
Similarly to the tip above you can run this baseline in parallel with the
``--dask-client`` option.
To understand the settings of this baseline PAD experiment you can check the
corresponding configuration file: ``bob/pad/face/config/lbp_svm.py``
......@@ -228,21 +261,37 @@ following command:
.. code-block:: sh
bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
$ bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/scores-dev.csv \
<PATH_TO_STORE_THE_RESULTS>/scores-eval.csv \
--legends "LBP features of facial region + SVM classifier + Replay-Mobile database" \
-e \
--eval \
--criterion eer \
-o <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
--output <PATH_TO_STORE_THE_RESULTS>/evaluation_report.pdf
The EER/HTER errors for the `Replay-Mobile`_ database are summarized in the table below:
+-------------------+----------+----------+
| Protocol | EER,\% | HTER,\% |
+===================+==========+==========+
| ``grandtest`` | 13.814 | 17.174 |
+-------------------+----------+----------+
=================== ================= =================
.. Development Evaluation
=================== ================= =================
APCER (mattescreen) 15.60% 25.77%
APCER (print) 12.97% 8.44%
APCER_AP 15.60% 25.77%
BPCER 14.29% 20.03%
ACER 14.94% 22.90%
FTA 0.00% 0.00%
FPR 14.28% (728/5098) 17.02% (647/3802)
FNR 14.29% (457/3199) 20.03% (439/2192)
HTER 14.28% 18.52%
FAR 14.28% 17.02%
FRR 14.29% 20.03%
PRECISION 0.79 0.73
RECALL 0.86 0.80
F1_SCORE 0.82 0.76
AUC 0.93 0.88
AUC-LOG-SCALE 1.83 1.76
=================== ================= =================
The ROC curves for the particular experiment can be downloaded from here:
......@@ -266,7 +315,8 @@ To run this baseline on the `Replay-Mobile`_ database, using the ``grandtest`` p
.. tip::
Similarly to the tip above you can run this baseline in parallel.
Similarly to the tip above you can run this baseline in parallel with the
``--dask-client`` option.
To understand the settings of this baseline PAD experiment you can check the
corresponding configuration file: ``bob/pad/face/config/qm_svm.py``
......@@ -276,21 +326,37 @@ following command:
.. code-block:: sh
bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
$ bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/scores-dev.csv \
<PATH_TO_STORE_THE_RESULTS>/scores-eval.csv \
--legends "IQM features of facial region + SVM classifier + Replay-Mobile database" \
-e \
--eval \
--criterion eer \
-o <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
--output <PATH_TO_STORE_THE_RESULTS>/evaluation_report.pdf
The EER/HTER errors for the `Replay-Mobile`_ database are summarized in the table below:
+-------------------+----------+----------+
| Protocol | EER,\% | HTER,\% |
+===================+==========+==========+
| ``grandtest`` | 1.747 | 4.074 |
+-------------------+----------+----------+
=================== ================ =================
.. Development Evaluation
=================== ================ =================
APCER (mattescreen) 1.54% 0.16%
APCER (print) 8.16% 9.53%
APCER_AP 8.16% 9.53%
BPCER 4.88% 10.77%
ACER 6.52% 10.15%
FTA 0.00% 0.00%
FPR 4.86% (248/5098) 4.89% (186/3802)
FNR 4.88% (156/3199) 10.77% (236/2192)
HTER 4.87% 7.83%
FAR 4.86% 4.89%
FRR 4.88% 10.77%
PRECISION 0.92 0.91
RECALL 0.95 0.89
F1_SCORE 0.94 0.90
AUC 0.99 0.98
AUC-LOG-SCALE 2.54 2.52
=================== ================ =================
The ROC curves for the particular experiment can be downloaded from here:
......@@ -299,84 +365,4 @@ The ROC curves for the particular experiment can be downloaded from here:
------------
.. _bob.pad.face.baselines.other_db:
Baselines on other databases
--------------------------------------
This section summarizes the results of baseline face PAD experiments on other databases.
------------
MIFS database + LBP features of facial region + SVM classifier
========================================================================
To run this baseline on the MIFS database, using the ``grandtest`` protocol, execute the following:
.. code-block:: sh
$ bob pad vanilla-pad mifs lbp svm-frames \
--output <PATH_TO_STORE_THE_RESULTS>
To evaluate the results computing EER, HTER and plotting ROC you can use the
following command:
.. code-block:: sh
bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
--legends "LBP features of facial region + SVM classifier + MIFS database" \
-e \
--criterion eer \
-o <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
The EER/HTER errors for the MIFS database are summarized in the table below:
+-------------------+----------+----------+
| Protocol | EER,\% | HTER,\% |
+===================+==========+==========+
| ``grandtest`` | 27.143 | 32.353 |
+-------------------+----------+----------+
------------
MIFS database + Image Quality Measures as features of facial region + SVM classifier
========================================================================================
To run this baseline on the MIFS database, using the ``grandtest`` protocol, execute the following:
.. code-block:: sh
$ bob pad vanilla-pad mifs qm svm-frames \
--output <PATH_TO_STORE_THE_RESULTS>
To evaluate the results computing EER, HTER and plotting ROC you can use the
following command:
.. code-block:: sh
bob pad evaluate \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev \
<PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
--legends "IQM features of facial region + SVM classifier + MIFS database" \
-e \
--criterion eer \
-o <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
The EER/HTER errors for the MIFS database are summarized in the table below:
+-------------------+----------+----------+
| Protocol | EER,\% | HTER,\% |
+===================+==========+==========+
| ``grandtest`` | 34.286 | 28.676 |
+-------------------+----------+----------+
------------
.. include:: links.rst
doc/img/ROC_iqm_svm_replay_attack.pdf
+ 0
− 0
View file @ 44eb214b
No preview for this file type
doc/img/ROC_iqm_svm_replay_mobile.pdf
+ 0
− 0
View file @ 44eb214b
No preview for this file type
doc/img/ROC_lbp_svm_replay_attack.pdf
+ 0
− 0
View file @ 44eb214b
No preview for this file type
doc/img/ROC_lbp_svm_replay_mobile.pdf
+ 0
− 0
View file @ 44eb214b
No preview for this file type
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment