Initial database

.gitignore

+*~
+*.swp
+*.pyc
+bin
+eggs
+parts
+.installed.cfg
+.mr.developer.cfg
+*.egg-info
+src
+develop-eggs
+sphinx
+dist
+*.sql3
+20170512-110547/

.gitlab-ci.yml

+test:
+  variables:
+    CONDA_ENVS_PATH: "conda-env"
+    CONDA_BLD_PATH: "conda-env"
+  script:
+    - hash -r
+    - conda config --set always_yes yes --set changeps1 no
+    - conda info -a
+    # https://www.idiap.ch does not work on CI machines but works elsewhere
+    - sed -i "s|https://www.idiap.ch|http://www.idiap.ch|g" environment.yml
+    - conda env create -vvv --file environment.yml
+    - source activate bob.paper.cvpr2018_facevuln
+    # this is a private link (works on the CI machine only)
+    - curl -o 20170512-110547.zip http://www.idiap.ch/private/wheels/gitlab/facenet_model2_20170512-110547.zip
+    - unzip 20170512-110547.zip
+    - buildout
+    - bin/bob_dbmanage.py all download --missing
+    - bin/nosetests -sv .
+  cache:
+    key: "$CI_BUILD_NAME"
+    paths:
+      - conda-env/.pkgs/*.tar.bz2
+      - conda-env/.pkgs/urls.txt
+      - conda-env/src_cache
+  image: continuumio/miniconda
+  tags:
+    - docker

MANIFEST.in

+include README.rst buildout.cfg COPYING version.txt requirements.txt
+recursive-include doc *.py *.rst *.ico *.png

README.rst

+.. vim: set fileencoding=utf-8 :
+.. Mon Sep 11 10:23:39 CEST 2017
+
+=======================================================================
+ A Study of the Robustness of Face Recognition to Presentation Attacks
+=======================================================================
+
+This package is part of the signal-processing and machine learning toolbox
+Bob_. It contains the source code to reproduce the following paper::
+
+
+    @article{Mohammadi_Deeply_Vulnerable_2017,
+              title = {Deeply Vulnerable -- A Study of the Robustness of Face Recognition to Presentation Attacks},
+             author = {Mohammadi, Amir and Bhattacharjee, Sushil and Marcel, S{\'{e}}bastien},
+            journal = {IET biometrics},
+               year = {2017},
+        institution = {Idiap},
+    }
+
+
+Installation
+------------
+
+The installation instructions are based on conda_ and works on **Linux systems
+only**. `Install conda`_ before continuing.
+
+Once you have installed conda_, download the source code of this paper and
+unpack it.  Then, you can create a conda environment with the following
+command::
+
+    $ cd bob.paper.cvpr2018_facevuln
+    $ conda env create -f environment.yml
+    $ source activate bob.paper.cvpr2018_facevuln  # activate the environment
+    $ buildout
+    $ bin/bob_dbmanage.py all download --missing
+    $ bin/nosetests -sv .  # test the installation
+
+This will install all the required software to reproduce this paper.
+
+Downloading the datasets
+------------------------
+
+Four datasets are used in this study which are publicly available.
+To download the datasets please refer to their websites:
+
+* `Mobio`_
+* `Replay Attack`_
+* `Replay Mobile`_
+* `MSU MFSD`_
+
+Downloading the face recognition models
+---------------------------------------
+
+Pre-trained face recognition (FR) models can be downloaded from their
+respective website as well.
+
+* `VGG Face`_ (`VGG Face model`_)
+* `LightCNN`_ (`LightCNN model`_)
+* `FaceNet`_ (`FaceNet model`_)
+
+The code will automatically download the VGG Face and LightCNN models and will
+place them in the right place. So you may skip downloading those manually.
+Please unzip the `FaceNet model`_ in the source directory with the name
+``20170512-110547``.
+
+ROC SDK experiments
+-------------------
+`ROC SDK`_ version 1.9 is used in the paper.
+The `ROC SDK`_ is a commercial product and a license is needed to use
+it. You can skip the `ROC SDK`_ experiments if you do not have a license.
+If you do, you will also need the Bob wrapper packages for using the ROC SDK.
+These packages are available upon request.
+
+Configuring the experiments
+---------------------------
+
+Now that you have downloaded the four databases. You need to set the paths to
+those in the configuration files. Bob_ supports a configuration file
+(``~/.bob_bio_databases.txt``) in your home directory to specify where the
+databases are located. Please specify the paths for the database like below::
+
+   $ cat ~/.bob_bio_databases.txt
+   [YOUR_MOBIO_IMAGE_DIRECTORY] = /databases/mobio/IMAGES_PNG/
+   [YOUR_MOBIO_ANNOTATION_DIRECTORY] = /databases/mobio/IMAGE_ANNOTATIONS/
+   [YOUR_MSU_MFSD_MOD_DIRECTORY] = /databases/MSU-MFSD/scene01/
+   [YOUR_REPLAY_ATTACK_DIRECTORY] = /databases/replay/protocols/replayattack-database/
+   [YOUR_REPLAY_MOBILE_DIRECTORY] = /databases/replay-mobile/database/
+
+Running the experiments
+-----------------------
+
+Follow the commands below to run the computational part of the experiments with
+`bob.bio.base`_.
+
+VGG::
+
+   $ bin/verify.py config_base.py config_cnn_eyes.py config_mobio.py
+   $ bin/verify.py config_base.py config_cnn.py config_replay_licit.py
+   $ bin/verify.py config_base.py config_cnn.py config_replay_spoof.py
+   $ bin/verify.py config_base.py config_cnn.py config_replaymobile_licit.py
+   $ bin/verify.py config_base.py config_cnn.py config_replaymobile_spoof.py
+   $ bin/verify.py config_base.py config_cnn_eyes.py config_msumfsd_licit.py
+   $ bin/verify.py config_base.py config_cnn_eyes.py config_msumfsd_spoof.py
+
+LightCNN::
+
+   $ bin/verify.py config_base.py config_lightcnn_eyes.py config_mobio.py
+   $ bin/verify.py config_base.py config_lightcnn.py config_replay_licit.py
+   $ bin/verify.py config_base.py config_lightcnn.py config_replay_spoof.py
+   $ bin/verify.py config_base.py config_lightcnn.py config_replaymobile_licit.py
+   $ bin/verify.py config_base.py config_lightcnn.py config_replaymobile_spoof.py
+   $ bin/verify.py config_base.py config_lightcnn_eyes.py config_msumfsd_licit.py
+   $ bin/verify.py config_base.py config_lightcnn_eyes.py config_msumfsd_spoof.py
+
+FaceNet::
+
+   $ bin/verify.py config_base.py config_facenet_eyes.py config_mobio.py
+   $ bin/verify.py config_base.py config_facenet.py config_replay_licit.py
+   $ bin/verify.py config_base.py config_facenet.py config_replay_spoof.py
+   $ bin/verify.py config_base.py config_facenet.py config_replaymobile_licit.py
+   $ bin/verify.py config_base.py config_facenet.py config_replaymobile_spoof.py
+   $ bin/verify.py config_base.py config_facenet_eyes.py config_msumfsd_licit.py
+   $ bin/verify.py config_base.py config_facenet_eyes.py config_msumfsd_spoof.py
+
+ISV::
+
+   $ bin/verify_isv.py config_base.py config_isv_eyes.py config_mobio.py -s isv-male
+   $ bin/verify_isv.py config_base.py config_isv_topleft.py config_replay_licit.py
+   $ bin/verify_isv.py config_base.py config_isv_topleft.py config_replay_spoof.py
+   $ bin/verify_isv.py config_base.py config_isv_topleft.py config_replaymobile_licit.py
+   $ bin/verify_isv.py config_base.py config_isv_topleft.py config_replaymobile_spoof.py
+   $ bin/verify_isv.py config_base.py config_isv_eyes.py config_msumfsd_licit.py
+   $ bin/verify_isv.py config_base.py config_isv_eyes.py config_msumfsd_spoof.py
+
+RANKONE::
+
+   $ bin/verify.py config_base.py config_rankone.py config_mobio.py
+   $ bin/verify.py config_base.py config_rankone_loaded.py config_replay_licit.py
+   $ bin/verify.py config_base.py config_rankone_loaded.py config_replay_spoof.py
+   $ bin/verify.py config_base.py config_rankone_loaded.py config_replaymobile_licit.py
+   $ bin/verify.py config_base.py config_rankone_loaded.py config_replaymobile_spoof.py
+   $ bin/verify.py config_base.py config_rankone_loaded.py config_msumfsd_licit.py
+   $ bin/verify.py config_base.py config_rankone_loaded.py config_msumfsd_spoof.py
+
+You can look at ``bin/verify.py --help`` to find options for parallelization of
+the runs. By default results are saved in the ``./results`` directory::
+
+   $ export RESULTS_DIR=./results
+
+The score files (output of ``verify.py`` commands) are also provided if you do
+not want to run all the experiments again. You can extract them from
+``./results/scores.tar.xz``. The score files can be used to generate the tables
+and figures.
+
+Generating the figures and tables
+---------------------------------
+
+The ROC and EPC curves on the Mobio dataset::
+
+   $ bin/evaluate.py -vvv \
+     -M 1e-5 -L 1e-3 -c EER \
+     -R mobio_nonorm_roc.pdf \
+     -E mobio_nonorm_epc.pdf \
+     -l VGG-Face LightCNN FaceNet ROC-SDK ISV \
+     -T "" "" \
+     -d \
+     $RESULTS_DIR/mobio/vgg-cosine/male/nonorm/scores-dev \
+     $RESULTS_DIR/mobio/lightcnn-cosine/male/nonorm/scores-dev \
+     $RESULTS_DIR/mobio/facenet-cosine/male/nonorm/scores-dev \
+     $RESULTS_DIR/mobio/rankone/male/nonorm/scores-dev \
+     $RESULTS_DIR/mobio/isv-male/male/nonorm/scores-dev \
+     -e \
+     $RESULTS_DIR/mobio/vgg-cosine/male/nonorm/scores-eval \
+     $RESULTS_DIR/mobio/lightcnn-cosine/male/nonorm/scores-eval \
+     $RESULTS_DIR/mobio/facenet-cosine/male/nonorm/scores-eval \
+     $RESULTS_DIR/mobio/rankone/male/nonorm/scores-eval \
+     $RESULTS_DIR/mobio/isv-male/male/nonorm/scores-eval
+
+Score distributions of ROC-SDK and VGG-Face on Mobio::
+
+   $ bin/threshold_evolution.py $RESULTS_DIR/mobio/rankone/male/nonorm/scores-{dev,eval} -o mobio_nonorm_rankone_hist.pdf
+   $ bin/threshold_evolution.py $RESULTS_DIR/mobio/vgg-cosine/male/nonorm/scores-{dev,eval} -o mobio_nonorm_vgg-cosine_hist.pdf
+
+Vulnerability analysis tables::
+
+   $ # concatenate scores of all three PA datasets:
+   $ for licit in "licit" "spoof";
+     do
+        for dev in "dev" "eval";
+        do
+           for alg in "facenet-cosine" "isv" "lightcnn-cosine" "rankone" "vgg-cosine";
+           do
+              cat $RESULTS_DIR/replay/$alg/grandtest-${licit}/nonorm/scores-${dev} \
+              $RESULTS_DIR/msu-mfsd-mod/$alg/grandtest-${licit}/nonorm/scores-${dev} \
+              $RESULTS_DIR/replay-mobile/$alg/grandtest-${licit}/nonorm/scores-${dev} \
+              > $RESULTS_DIR/three-$alg-${licit}-${dev};
+           done
+        done
+     done
+
+   $ # print the table values:
+   $ echo "replay attack"
+   $ for alg in "vgg-cosine" "lightcnn-cosine" "facenet-cosine" "rankone" "isv" ; do echo $alg && bin/vulnerability.py $RESULTS_DIR/replay/$alg/grandtest-{licit,spoof}/nonorm/scores-{dev,eval}; done
+   $ echo "msu mfsd"
+   $ for alg in "vgg-cosine" "lightcnn-cosine" "facenet-cosine" "rankone" "isv" ; do echo $alg && bin/vulnerability.py $RESULTS_DIR/msu-mfsd-mod/$alg/grandtest-{licit,spoof}/nonorm/scores-{dev,eval}; done
+   $ echo "replay mobile"
+   $ for alg in "vgg-cosine" "lightcnn-cosine" "facenet-cosine" "rankone" "isv" ; do echo $alg && bin/vulnerability.py $RESULTS_DIR/replay-mobile/$alg/grandtest-{licit,spoof}/nonorm/scores-{dev,eval}; done
+   $ echo "all datasets"
+   $ for alg in "vgg-cosine" "lightcnn-cosine" "facenet-cosine" "rankone" "isv" ; do echo $alg && bin/vulnerability.py $RESULTS_DIR/three-$alg-{licit,spoof}-{dev,eval}; done
+
+Score distribution of all three PA datasets combined::
+
+   $ bin/plot_on_demand_better.py -v $RESULTS_DIR/three-{vgg-cosine,lightcnn-cosine,facenet-cosine,rankone,isv}-{licit,spoof}-{dev,eval}  -t VGG-Face,LightCNN,FaceNet,ROC-SDK,ISV -o allthree_dist.pdf
+
+
+Contact
+-------
+
+For questions or reporting issues to this software package, contact our
+development `mailing list`_.
+
+
+.. Place your references here:
+.. _bob: https://www.idiap.ch/software/bob
+.. _mailing list: https://www.idiap.ch/software/bob/discuss
+.. _conda: https://conda.io
+.. _install conda: https://conda.io/docs/install/quick.html#linux-miniconda-install
+.. _bob.bio.base: https://pypi.python.org/pypi/bob.bio.base
+.. _paper: http://publications.idiap.ch/index.php/publications/show/$$$$
+.. _mobio: https://www.idiap.ch/dataset/mobio
+.. _replay attack: http://www.idiap.ch/dataset/replayattack
+.. _replay mobile: http://www.idiap.ch/dataset/replay-mobile
+.. _msu mfsd: https://www.cse.msu.edu/rgroups/biometrics/Publications/Databases/MSUMobileFaceSpoofing/index.htm
+.. _vgg face: http://www.robots.ox.ac.uk/~vgg/software/vgg_face
+.. _vgg face model: http://www.robots.ox.ac.uk/~vgg/software/vgg_face
+.. _lightcnn: https://github.com/AlfredXiangWu/face_verification_experiment
+.. _lightcnn model: https://github.com/AlfredXiangWu/face_verification_experiment/archive/master.zip
+.. _facenet: https://github.com/davidsandberg/facenet
+.. _facenet model: https://drive.google.com/uc?export=download&confirm=YCEV&id=0B5MzpY9kBtDVZ2RpVDYwWmxoSUk
+.. _roc sdk: https://www.rankone.io

bob/__init__.py

+# see https://docs.python.org/3/library/pkgutil.html
+from pkgutil import extend_path
+__path__ = extend_path(__path__, __name__)

bob/paper/__init__.py

+# see https://docs.python.org/3/library/pkgutil.html
+from pkgutil import extend_path
+__path__ = extend_path(__path__, __name__)

bob/paper/cvpr2018_facevuln/__init__.py

+from . import script
+from .utils import *
+
+
+def get_config():
+    """Returns a string containing the configuration information.
+    """
+
+    import bob.extension
+    return bob.extension.get_config(__name__)
+
+
+# gets sphinx autodoc done right - don't remove it
+__all__ = [_ for _ in dir() if not _.startswith('_')]

bob/paper/cvpr2018_facevuln/database/__init__.py

+from .query import Database
+
+
+def __appropriate__(*args):
+    """Says object was actually declared here, and not in the import module.
+    Fixing sphinx warnings of not being able to find classes, when path is
+    shortened. Parameters:
+
+      *args: An iterable of objects to modify
+
+    Resolves `Sphinx referencing issues
+    <https://github.com/sphinx-doc/sphinx/issues/3048>`
+    """
+
+    for obj in args:
+        obj.__module__ = __name__
+
+
+__appropriate__(
+    Database,
+)
+# gets sphinx autodoc done right - don't remove it
+__all__ = [_ for _ in dir() if not _.startswith('_')]

bob/paper/cvpr2018_facevuln/database/bonafide_to_zei.py

+#!/usr/bin/env python
+"""Takes a list bona-fide files and generates their zei list.
+Usage:
+  %(prog)s <infile> <outfile>
+  %(prog)s -h | --help
+
+Options:
+  -h --help     Show this screen.
+
+"""
+from collections import defaultdict
+import six
+
+
+def convert(infile, outfile):
+    bonafides = defaultdict(list)
+    with open(infile, 'rt') as f:
+        for line in f:
+            path, client_id = line.split()
+            bonafides[client_id].append(path)
+    zeis = defaultdict(list)
+    all_clients = list(bonafides.keys())
+    for bf_client_id, paths in bonafides.items():
+        for client_id in all_clients:
+            if client_id == bf_client_id:
+                continue
+            zeis[client_id] += six.moves.zip_longest(
+                paths, [bf_client_id], fillvalue=bf_client_id)
+    with open(outfile, 'wt') as f:
+        for client_id, paths in zeis.items():
+            for path, bf_client_id in paths:
+                f.write('{0} {1} {1} {2}\n'.format(
+                    path, client_id, bf_client_id))
+
+
+def main():
+    import sys
+    import os
+    import docopt
+
+    completions = dict(
+        prog=os.path.basename(sys.argv[0]),
+    )
+
+    args = docopt.docopt(
+        __doc__ % completions,
+    )
+
+    convert(args['<infile>'], args['<outfile>'])
+
+
+if __name__ == '__main__':
+    main()

bob/paper/cvpr2018_facevuln/database/config.py

+from bob.paper.cvpr2018_facevuln.database import Database
+
+ORIGINAL_DIRECTORY = '[SILICONECVPR_3DMask_Data_DIRECTORY]'
+database_licit = Database(original_directory=ORIGINAL_DIRECTORY, protocol='licit')
+database_spoof = Database(original_directory=ORIGINAL_DIRECTORY, protocol='spoof')

bob/paper/cvpr2018_facevuln/database/lists/licit/dev/bona_fides.lst

+Bonafide/A/photos/no_coop/DSCN3121.JPG A
+Bonafide/A/photos/with_coop/DSCN3115.JPG A
+Bonafide/B/photos/no_coop/DSCN3100.JPG B
+Bonafide/B/photos/with_coop/DSCN3095.JPG B
+Bonafide/C/photos/no_coop/DSCN3188.JPG C
+Bonafide/C/photos/with_coop/DSCN3167.JPG C
+Bonafide/C/photos/with_coop/DSCN3192.JPG C
+Bonafide/D/photos/with_coop/DSCN3105.JPG D
+Bonafide/E/photos/no_coop/DSCN3148.JPG E
+Bonafide/E/photos/no_coop/DSCN3149.JPG E
+Bonafide/E/photos/no_coop/DSCN3150.JPG E
+Bonafide/E/photos/no_coop/DSCN3151.JPG E
+Bonafide/E/photos/no_coop/DSCN3152.JPG E
+Bonafide/E/photos/with_coop/DSCN3142.JPG E
+Bonafide/F/photos/no_coop/DSCN3131.JPG F
+Bonafide/F/photos/no_coop/DSCN3132.JPG F
+Bonafide/F/photos/with_coop/DSCN3126.JPG F
+Mask_Batl_Data/bonafide/E_gen_i0_003.h5 E
+Mask_Batl_Data/bonafide/E_gen_i1_004.h5 E
+Mask_Batl_Data/bonafide/E_gen_i2_005.h5 E
+Mask_Batl_Data/bonafide/E_gen_i3_006.h5 E
+Mask_Batl_Data/bonafide/F_gen_i0_007.h5 F
+Mask_Batl_Data/bonafide/F_gen_i1_008.h5 F
+Mask_Batl_Data/bonafide/F_gen_i2_009.h5 F
+Mask_Batl_Data/bonafide/F_gen_i3_010.h5 F
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i0_061.h5 A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i1_062.h5 A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i2_063.h5 A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i3_064.h5 A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/B_gen_i0_006.h5 B
+Mask_Batl_Data4_BonaFide/captured/no_glasses/B_gen_i3_002.h5 B
+Mask_Batl_Data4_BonaFide/captured/no_glasses/B_Gen_i2_001.h5 B
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i0_001.h5 D
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i1_003.h5 D
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i2_004.h5 D
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i3_005.h5 D
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i0_005.h5 B
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i1_006.h5 B
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i2_007.h5 B
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i3_008.h5 B
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i0_001.h5 D
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i1_002.h5 D
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i2_003.h5 D
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i3_004.h5 D

bob/paper/cvpr2018_facevuln/database/lists/licit/dev/for_models.lst

+Bonafide/A/photos/with_coop/DSCN3115.JPG A A
+Bonafide/B/photos/with_coop/DSCN3095.JPG B B
+Bonafide/C/photos/with_coop/DSCN3167.JPG C C
+Bonafide/C/photos/with_coop/DSCN3192.JPG C C
+Bonafide/D/photos/with_coop/DSCN3105.JPG D D
+Bonafide/E/photos/with_coop/DSCN3142.JPG E E
+Bonafide/F/photos/with_coop/DSCN3126.JPG F F

bob/paper/cvpr2018_facevuln/database/lists/licit/dev/for_scores.lst

+Bonafide/A/photos/no_coop/DSCN3121.JPG A A A
+Bonafide/B/photos/no_coop/DSCN3100.JPG B B B
+Bonafide/C/photos/no_coop/DSCN3188.JPG C C C
+Bonafide/E/photos/no_coop/DSCN3148.JPG E E E
+Bonafide/E/photos/no_coop/DSCN3149.JPG E E E
+Bonafide/E/photos/no_coop/DSCN3150.JPG E E E
+Bonafide/E/photos/no_coop/DSCN3151.JPG E E E
+Bonafide/E/photos/no_coop/DSCN3152.JPG E E E
+Bonafide/F/photos/no_coop/DSCN3131.JPG F F F
+Bonafide/F/photos/no_coop/DSCN3132.JPG F F F
+Mask_Batl_Data/bonafide/E_gen_i0_003.h5 E E E
+Mask_Batl_Data/bonafide/E_gen_i1_004.h5 E E E
+Mask_Batl_Data/bonafide/E_gen_i2_005.h5 E E E
+Mask_Batl_Data/bonafide/E_gen_i3_006.h5 E E E
+Mask_Batl_Data/bonafide/F_gen_i0_007.h5 F F F
+Mask_Batl_Data/bonafide/F_gen_i1_008.h5 F F F
+Mask_Batl_Data/bonafide/F_gen_i2_009.h5 F F F
+Mask_Batl_Data/bonafide/F_gen_i3_010.h5 F F F
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i0_061.h5 A A A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i1_062.h5 A A A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i2_063.h5 A A A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i3_064.h5 A A A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/B_gen_i0_006.h5 B B B
+Mask_Batl_Data4_BonaFide/captured/no_glasses/B_gen_i3_002.h5 B B B
+Mask_Batl_Data4_BonaFide/captured/no_glasses/B_Gen_i2_001.h5 B B B
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i0_001.h5 D D D
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i1_003.h5 D D D
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i2_004.h5 D D D
+Mask_Batl_Data4_BonaFide/captured/no_glasses/D_gen_i3_005.h5 D D D
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i0_005.h5 B B B
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i1_006.h5 B B B
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i2_007.h5 B B B
+Mask_Batl_Data4_BonaFide/captured/withglasses/B_genglasses_i3_008.h5 B B B
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i0_001.h5 D D D
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i1_002.h5 D D D
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i2_003.h5 D D D
+Mask_Batl_Data4_BonaFide/captured/withglasses/D_genglasses_i3_004.h5 D D D
+Bonafide/A/photos/no_coop/DSCN3121.JPG B B A
+Bonafide/A/photos/no_coop/DSCN3121.JPG C C A
+Bonafide/A/photos/no_coop/DSCN3121.JPG D D A
+Bonafide/A/photos/no_coop/DSCN3121.JPG E E A
+Bonafide/A/photos/no_coop/DSCN3121.JPG F F A
+Bonafide/A/photos/with_coop/DSCN3115.JPG B B A
+Bonafide/A/photos/with_coop/DSCN3115.JPG C C A
+Bonafide/A/photos/with_coop/DSCN3115.JPG D D A
+Bonafide/A/photos/with_coop/DSCN3115.JPG E E A
+Bonafide/A/photos/with_coop/DSCN3115.JPG F F A
+Bonafide/B/photos/no_coop/DSCN3100.JPG A A B
+Bonafide/B/photos/no_coop/DSCN3100.JPG C C B
+Bonafide/B/photos/no_coop/DSCN3100.JPG D D B
+Bonafide/B/photos/no_coop/DSCN3100.JPG E E B
+Bonafide/B/photos/no_coop/DSCN3100.JPG F F B
+Bonafide/B/photos/with_coop/DSCN3095.JPG A A B
+Bonafide/B/photos/with_coop/DSCN3095.JPG C C B
+Bonafide/B/photos/with_coop/DSCN3095.JPG D D B
+Bonafide/B/photos/with_coop/DSCN3095.JPG E E B
+Bonafide/B/photos/with_coop/DSCN3095.JPG F F B
+Bonafide/C/photos/no_coop/DSCN3188.JPG A A C
+Bonafide/C/photos/no_coop/DSCN3188.JPG B B C
+Bonafide/C/photos/no_coop/DSCN3188.JPG D D C
+Bonafide/C/photos/no_coop/DSCN3188.JPG E E C
+Bonafide/C/photos/no_coop/DSCN3188.JPG F F C
+Bonafide/C/photos/with_coop/DSCN3167.JPG A A C
+Bonafide/C/photos/with_coop/DSCN3167.JPG B B C
+Bonafide/C/photos/with_coop/DSCN3167.JPG D D C
+Bonafide/C/photos/with_coop/DSCN3167.JPG E E C
+Bonafide/C/photos/with_coop/DSCN3167.JPG F F C
+Bonafide/C/photos/with_coop/DSCN3192.JPG A A C
+Bonafide/C/photos/with_coop/DSCN3192.JPG B B C
+Bonafide/C/photos/with_coop/DSCN3192.JPG D D C
+Bonafide/C/photos/with_coop/DSCN3192.JPG E E C
+Bonafide/C/photos/with_coop/DSCN3192.JPG F F C
+Bonafide/D/photos/with_coop/DSCN3105.JPG A A D
+Bonafide/D/photos/with_coop/DSCN3105.JPG B B D
+Bonafide/D/photos/with_coop/DSCN3105.JPG C C D
+Bonafide/D/photos/with_coop/DSCN3105.JPG E E D
+Bonafide/D/photos/with_coop/DSCN3105.JPG F F D
+Bonafide/E/photos/no_coop/DSCN3148.JPG A A E
+Bonafide/E/photos/no_coop/DSCN3148.JPG B B E
+Bonafide/E/photos/no_coop/DSCN3148.JPG C C E
+Bonafide/E/photos/no_coop/DSCN3148.JPG D D E
+Bonafide/E/photos/no_coop/DSCN3148.JPG F F E
+Bonafide/E/photos/no_coop/DSCN3149.JPG A A E
+Bonafide/E/photos/no_coop/DSCN3149.JPG B B E
+Bonafide/E/photos/no_coop/DSCN3149.JPG C C E
+Bonafide/E/photos/no_coop/DSCN3149.JPG D D E
+Bonafide/E/photos/no_coop/DSCN3149.JPG F F E
+Bonafide/E/photos/no_coop/DSCN3150.JPG A A E
+Bonafide/E/photos/no_coop/DSCN3150.JPG B B E
+Bonafide/E/photos/no_coop/DSCN3150.JPG C C E
+Bonafide/E/photos/no_coop/DSCN3150.JPG D D E
+Bonafide/E/photos/no_coop/DSCN3150.JPG F F E
+Bonafide/E/photos/no_coop/DSCN3151.JPG A A E
+Bonafide/E/photos/no_coop/DSCN3151.JPG B B E
+Bonafide/E/photos/no_coop/DSCN3151.JPG C C E
+Bonafide/E/photos/no_coop/DSCN3151.JPG D D E
+Bonafide/E/photos/no_coop/DSCN3151.JPG F F E
+Bonafide/E/photos/no_coop/DSCN3152.JPG A A E
+Bonafide/E/photos/no_coop/DSCN3152.JPG B B E
+Bonafide/E/photos/no_coop/DSCN3152.JPG C C E
+Bonafide/E/photos/no_coop/DSCN3152.JPG D D E
+Bonafide/E/photos/no_coop/DSCN3152.JPG F F E
+Bonafide/E/photos/with_coop/DSCN3142.JPG A A E
+Bonafide/E/photos/with_coop/DSCN3142.JPG B B E
+Bonafide/E/photos/with_coop/DSCN3142.JPG C C E
+Bonafide/E/photos/with_coop/DSCN3142.JPG D D E
+Bonafide/E/photos/with_coop/DSCN3142.JPG F F E
+Bonafide/F/photos/no_coop/DSCN3131.JPG A A F
+Bonafide/F/photos/no_coop/DSCN3131.JPG B B F
+Bonafide/F/photos/no_coop/DSCN3131.JPG C C F
+Bonafide/F/photos/no_coop/DSCN3131.JPG D D F
+Bonafide/F/photos/no_coop/DSCN3131.JPG E E F
+Bonafide/F/photos/no_coop/DSCN3132.JPG A A F
+Bonafide/F/photos/no_coop/DSCN3132.JPG B B F
+Bonafide/F/photos/no_coop/DSCN3132.JPG C C F
+Bonafide/F/photos/no_coop/DSCN3132.JPG D D F
+Bonafide/F/photos/no_coop/DSCN3132.JPG E E F
+Bonafide/F/photos/with_coop/DSCN3126.JPG A A F
+Bonafide/F/photos/with_coop/DSCN3126.JPG B B F
+Bonafide/F/photos/with_coop/DSCN3126.JPG C C F
+Bonafide/F/photos/with_coop/DSCN3126.JPG D D F
+Bonafide/F/photos/with_coop/DSCN3126.JPG E E F
+Mask_Batl_Data/bonafide/E_gen_i0_003.h5 A A E
+Mask_Batl_Data/bonafide/E_gen_i0_003.h5 B B E
+Mask_Batl_Data/bonafide/E_gen_i0_003.h5 C C E
+Mask_Batl_Data/bonafide/E_gen_i0_003.h5 D D E
+Mask_Batl_Data/bonafide/E_gen_i0_003.h5 F F E
+Mask_Batl_Data/bonafide/E_gen_i1_004.h5 A A E
+Mask_Batl_Data/bonafide/E_gen_i1_004.h5 B B E
+Mask_Batl_Data/bonafide/E_gen_i1_004.h5 C C E
+Mask_Batl_Data/bonafide/E_gen_i1_004.h5 D D E
+Mask_Batl_Data/bonafide/E_gen_i1_004.h5 F F E
+Mask_Batl_Data/bonafide/E_gen_i2_005.h5 A A E
+Mask_Batl_Data/bonafide/E_gen_i2_005.h5 B B E
+Mask_Batl_Data/bonafide/E_gen_i2_005.h5 C C E
+Mask_Batl_Data/bonafide/E_gen_i2_005.h5 D D E
+Mask_Batl_Data/bonafide/E_gen_i2_005.h5 F F E
+Mask_Batl_Data/bonafide/E_gen_i3_006.h5 A A E
+Mask_Batl_Data/bonafide/E_gen_i3_006.h5 B B E
+Mask_Batl_Data/bonafide/E_gen_i3_006.h5 C C E
+Mask_Batl_Data/bonafide/E_gen_i3_006.h5 D D E
+Mask_Batl_Data/bonafide/E_gen_i3_006.h5 F F E
+Mask_Batl_Data/bonafide/F_gen_i0_007.h5 A A F
+Mask_Batl_Data/bonafide/F_gen_i0_007.h5 B B F
+Mask_Batl_Data/bonafide/F_gen_i0_007.h5 C C F
+Mask_Batl_Data/bonafide/F_gen_i0_007.h5 D D F
+Mask_Batl_Data/bonafide/F_gen_i0_007.h5 E E F
+Mask_Batl_Data/bonafide/F_gen_i1_008.h5 A A F
+Mask_Batl_Data/bonafide/F_gen_i1_008.h5 B B F
+Mask_Batl_Data/bonafide/F_gen_i1_008.h5 C C F
+Mask_Batl_Data/bonafide/F_gen_i1_008.h5 D D F
+Mask_Batl_Data/bonafide/F_gen_i1_008.h5 E E F
+Mask_Batl_Data/bonafide/F_gen_i2_009.h5 A A F
+Mask_Batl_Data/bonafide/F_gen_i2_009.h5 B B F
+Mask_Batl_Data/bonafide/F_gen_i2_009.h5 C C F
+Mask_Batl_Data/bonafide/F_gen_i2_009.h5 D D F
+Mask_Batl_Data/bonafide/F_gen_i2_009.h5 E E F
+Mask_Batl_Data/bonafide/F_gen_i3_010.h5 A A F
+Mask_Batl_Data/bonafide/F_gen_i3_010.h5 B B F
+Mask_Batl_Data/bonafide/F_gen_i3_010.h5 C C F
+Mask_Batl_Data/bonafide/F_gen_i3_010.h5 D D F
+Mask_Batl_Data/bonafide/F_gen_i3_010.h5 E E F
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i0_061.h5 B B A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i0_061.h5 C C A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i0_061.h5 D D A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i0_061.h5 E E A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i0_061.h5 F F A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i1_062.h5 B B A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i1_062.h5 C C A
+Mask_Batl_Data4_BonaFide/captured/no_glasses/A_gen_i1_062.h5 D D A