Updated FrameDifference preprocessor to deal with missing and excess...

Updated FrameDifference preprocessor to deal with missing and excess annotations + doc on baselines for MSU MFSD

bob/pad/face/preprocessor/FrameDifference.py

@@ -300,6 +300,57 @@ class FrameDifference(Preprocessor, object):
         return diff
 
 
+    #==========================================================================
+    def select_annotated_frames(self, frames, annotations):
+        """
+        Select only annotated frames in the input FrameContainer ``frames``.
+
+        **Parameters:**
+
+        ``frames`` : FrameContainer
+            Video data stored in the FrameContainer, see ``bob.bio.video.utils.FrameContainer``
+            for further details.
+
+        ``annotations`` : :py:class:`dict`
+            A dictionary containing the annotations for each frame in the video.
+            Dictionary structure: ``annotations = {'1': frame1_dict, '2': frame1_dict, ...}``.
+            Where ``frameN_dict = {'topleft': (row, col), 'bottomright': (row, col)}``
+            is the dictionary defining the coordinates of the face bounding box in frame N.
+
+        **Returns:**
+
+        ``cleaned_frame_container`` : FrameContainer
+            FrameContainer containing the annotated frames only.
+
+        ``cleaned_annotations`` : :py:class:`dict`
+            A dictionary containing the annotations for each frame in the output video.
+            Dictionary structure: ``annotations = {'1': frame1_dict, '2': frame1_dict, ...}``.
+            Where ``frameN_dict = {'topleft': (row, col), 'bottomright': (row, col)}``
+            is the dictionary defining the coordinates of the face bounding box in frame N.
+        """
+
+        annotated_frames = np.sort( [np.int(item) for item in annotations.keys()] ) # annotated frame numbers
+
+        available_frames = range(0,len(frames)) # frame numbers in the input video
+
+        valid_frames = list(set(annotated_frames).intersection(available_frames)) # valid and annotated frames
+
+        cleaned_frame_container = bob.bio.video.FrameContainer() # initialize the FrameContainer
+
+        cleaned_annotations = {}
+
+        for idx, valid_frame_num in enumerate(valid_frames):
+            ## valid_frame_num - is the number of the original frame having annotations
+
+            cleaned_annotations[str(idx)] = annotations[str(valid_frame_num)] # correct the frame numbers
+
+            selected_frame = frames[valid_frame_num][1] # get current frame
+
+            cleaned_frame_container.add(idx, selected_frame) # add current frame to FrameContainer
+
+        return cleaned_frame_container, cleaned_annotations
+
+
     #==========================================================================
     def __call__(self, frames, annotations):
         """
@@ -329,6 +380,11 @@ class FrameDifference(Preprocessor, object):
             The second column contains frame differences of non-facial/background regions.
         """
 
+        if len(frames) != len(annotations): # if some annotations are missing
+
+            ## Select only annotated frames:
+            frames, annotations = self.select_annotated_frames(frames, annotations)
+
         if self.check_face_size_flag:
 
             selected_frames, selected_annotations = self.check_face_size(frames, annotations, self.min_face_size)

doc/baselines.rst

@@ -395,6 +395,162 @@ The ROC curves for the particular experiment can be downloaded from here:
 
 ------------
 
+.. _bob.pad.face.baselines.msu_mfsd:
+
+Baselines on MSU MFSD database
+--------------------------------------
+
+This section summarizes the results of baseline face PAD experiments on the `MSU MFSD`_ database.
+The description of the database instance, which can be used to run face PAD experiments on the MSU MFSD is given
+here :ref:`bob.pad.face.resources.databases.msu_mfsd`.
+
+
+LBP features of facial region + SVM classifier
+========================================================================
+
+Detailed description of this PAD pipe-line is given at :ref:`bob.pad.face.resources.face_pad.lbp_svm_replayattack`.
+Note, that the same PAD pipe-line was used to run experiments on the Replay-Attack database.
+
+To run this baseline on the `MSU MFSD`_ database, using the ``grandtest`` protocol, execute the following:
+
+.. code-block:: sh
+
+    $ ./bin/spoof.py lbp-svm \
+    --database msu-mfsd --protocol grandtest --groups train dev eval \
+    --sub-directory <PATH_TO_STORE_THE_RESULTS>
+
+.. tip::
+
+    Similarly to the tip above you can run this baseline in parallel.
+
+To understand the settings of this baseline PAD experiment you can check the
+corresponding configuration file: ``bob/pad/face/config/lbp_svm.py``
+
+To evaluate the results computing EER, HTER and plotting ROC you can use the
+following command:
+
+.. code-block:: sh
+
+    ./bin/evaluate.py \
+    --dev-files  <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev  \
+    --eval-files <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
+    --legends "LBP features of facial region + SVM classifier + MSU MFSD database" \
+    -F 7 \
+    --criterion EER \
+    --roc <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
+
+The EER/HTER errors for the `MSU MFSD`_ database are summarized in the Table below:
+
++-------------------+----------+----------+
+|      Protocol     |  EER,\%  |  HTER,\% |
++===================+==========+==========+
+|   ``grandtest``   |  27.402  |  21.399  |
++-------------------+----------+----------+
+
+The ROC curves for the particular experiment can be downloaded from here:
+
+:download:`ROC curve <img/ROC_lbp_svm_msu_mfsd.pdf>`
+
+------------
+
+
+Image Quality Measures as features of facial region + SVM classifier
+========================================================================
+
+Detailed description of this PAD pipe-line is given at :ref:`bob.pad.face.resources.face_pad.qm_svm_replayattack`.
+Note, that the same PAD pipe-line was used to run experiments on the Replay-Attack database.
+
+To run this baseline on the `MSU MFSD`_ database, using the ``grandtest`` protocol, execute the following:
+
+.. code-block:: sh
+
+    $ ./bin/spoof.py qm-svm \
+    --database msu-mfsd --protocol grandtest --groups train dev eval \
+    --sub-directory <PATH_TO_STORE_THE_RESULTS>
+
+.. tip::
+
+    Similarly to the tip above you can run this baseline in parallel.
+
+To understand the settings of this baseline PAD experiment you can check the
+corresponding configuration file: ``bob/pad/face/config/qm_svm.py``
+
+To evaluate the results computing EER, HTER and plotting ROC you can use the
+following command:
+
+.. code-block:: sh
+
+    ./bin/evaluate.py \
+    --dev-files  <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev  \
+    --eval-files <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
+    --legends "IQM features of facial region + SVM classifier + MSU MFSD database" \
+    -F 7 \
+    --criterion EER \
+    --roc <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
+
+The EER/HTER errors for the `MSU MFSD`_ database are summarized in the Table below:
+
++-------------------+----------+----------+
+|      Protocol     |  EER,\%  |  HTER,\% |
++===================+==========+==========+
+|   ``grandtest``   |  4.115   |  5.564   |
++-------------------+----------+----------+
+
+The ROC curves for the particular experiment can be downloaded from here:
+
+:download:`ROC curve <img/ROC_iqm_svm_msu_mfsd.pdf>`
+
+------------
+
+
+Frame differences based features (motion analysis) + SVM classifier
+========================================================================
+
+Detailed description of this PAD pipe-line is given at :ref:`bob.pad.face.resources.face_pad.frame_diff_svm_replayattack`.
+Note, that the same PAD pipe-line was used to run experiments on the Replay-Attack database.
+
+To run this baseline on the `MSU MFSD`_ database, using the ``grandtest`` protocol, execute the following:
+
+.. code-block:: sh
+
+    $ ./bin/spoof.py frame-diff-svm \
+    --database msu-mfsd --protocol grandtest --groups train dev eval \
+    --sub-directory <PATH_TO_STORE_THE_RESULTS>
+
+.. tip::
+
+    Similarly to the tip above you can run this baseline in parallel.
+
+To understand the settings of this baseline PAD experiment you can check the
+corresponding configuration file: ``bob/pad/face/config/frame_diff_svm.py``
+
+To evaluate the results computing EER, HTER and plotting ROC you can use the
+following command:
+
+.. code-block:: sh
+
+    ./bin/evaluate.py \
+    --dev-files  <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-dev  \
+    --eval-files <PATH_TO_STORE_THE_RESULTS>/grandtest/scores/scores-eval \
+    --legends "10 features for each window in Frame Differences + SVM classifier + MSU MFSD database" \
+    -F 7 \
+    --criterion EER \
+    --roc <PATH_TO_STORE_THE_RESULTS>/ROC.pdf
+
+The EER/HTER errors for the `MSU MFSD`_ database are summarized in the Table below:
+
++-------------------+----------+----------+
+|      Protocol     |  EER,\%  |  HTER,\% |
++===================+==========+==========+
+|   ``grandtest``   |  25.839  |  17.050  |
++-------------------+----------+----------+
+
+The ROC curves for the particular experiment can be downloaded from here:
+
+:download:`ROC curve <img/ROC_frame_diff_svm_msu_mfsd.pdf>`
+
+------------
+
 
 .. include:: links.rst