Can personalised hygienic masks be used to attack face recognition systems?

This package is part of the signal-processing and machine learning toolbox Bob.

Click on here to go to the Bob website and here to visit the PyPI page.

This package contains source code to replicate the experimental results published in the following paper::

@inproceedings{Komaty_IJCB2023_2023,
     author = {Komaty, Alain and Krivokuca, Vedrana and Ecabert, Christophe and Marcel, S{\'{e}}bastien},
   projects = {SOTERIA},
      title = {Can personalised hygienic masks be used to attack face recognition systems?},
  booktitle = {Proceedings of IEEE International Joint Conference on Biometrics (IJCB2023)},
       year = {2023},
   abstract = {The proliferation of automated face recognition (FR) necessitates increasingly accurate person identification.  The COVID-19 pandemic has exposed the limitations of FR systems when presented with faces occluded by hygienic masks. However, the security risks of personalised hygienic mask attacks, whereby an attacker wears the mask on which the bottom part of an enrolled user's face is printed, have not yet been studied. To address this research gap, we introduce a novel face dataset consisting of smartphone-recorded videos of real (bona-fide) faces and personalised hygienic mask attacks. We also analyse the vulnerability of two state-of-the-art FR systems to this type of attack, using our dataset.  Our results indicate that personalised hygienic mask attacks have the potential to compromise system security, particularly for FR systems that are tuned towards optimising user convenience. These findings underscore the importance of developing suitable Presentation Attack Detection (PAD) algorithms. Our dataset will help researchers and practitioners work towards this goal, thereby enhancing the security and reliability of FR systems.},
        pdf = {https://publidiap.idiap.ch/attachments/papers/2023/Komaty_IJCB2023_2023.pdf}

}

If you use this package and/or its results, please consider citing the paper.

Installation

The installation instructions are based on conda and works on Linux/Mac systems only. Install conda before continuing.

Once you have installed conda, download the source code of this paper and unpack it. Then, install Bob using conda.

After install Bob successfully, run the following command:

pip install -r requirements.txt

Repository structure

This repository is organised as follows:

• database/: contains the codes used to organize the database in pandas dataframes. The dataframes will be then used by the pipeline to run the full experiment
• preprocessor/: contains the codes to extract the frames from the videos and the code for face cropping.
• pipeline_vuln.py: contains the codes to run the pipeline.

Running the pipeline

1. Download the database from the following link: https://www.idiap.ch/en/dataset/phymatt

2. Update conda using:

   mamba update -n base -c conda-forge conda mamba

3. Create a new conda environment using the provided environment file environment.yml:

   mamba env create -f environment.yml

4. Activate the environment:

   conda activate bob_hyg_mask

5. Create a list of the videos to be used for the experiment. The list should contain the path for each video you want in the experiment. If you want all videos, you can use the following command:

   find <path_to_database> -name "*.mp4" > <path_to_list>

6. Run the frames extraction code as follows:

   python preprocessor/extract_frames.py -l <path_to_list> -o <path_to_output_folder>`

7. Run the database organization code as follows:

   python database/create_database_dataframe.py --frames_list --output_path  -metadata_filename -save_mode --min_face_size`

8. Run the pipeline as follows:

   python pipeline_vuln.py --database_path --output_path --metadata_filename --save_mode --min_face_size --attack_type --attack_params --attac

9. Once you have the score files, namely the score-dev.csv, you can use the script utils/split_scores.sh to split the scores into bona-fide and attack scores. The script will create three files: scores-dev_print-attack.csv, scores-dev_replay-attack.csv and scores-dev_hyg-maks.csv.

10. You can then use these files to compute the metrics as follows:

    bob vuln metrics scores-dev_print-attack.csv scores-dev_replay-attack.csv scores-dev_hyg-maks.csv

Contact

For questions or reporting issues to this software package, contact our development mailing list.