Synthetic to Authentic

(ECCVw24 Oral)

Parsa Rahimi^1,2 Behrooz Razeghi² Sébastien Marcel²

¹ EPFL, ² Idiap

This is the code for our paper Syn2Auth. In this paper we studied the possibility of using different Image-to-Image transfer methodologies for making the images in the 3D rendered datasets (e.g., DigiFace1M) look more realistic while boosting the performance of the FR system which trained on the more realistic images. This is done without using any identity preservation prior (e.g., another strong FR system) or large datasets used for training FR systems.

News

• [2024/08/10] Our paper got accepted as oral presentation in ECCVw24.

General Setup

    conda create -f env.yaml
    conda activate syn2auth

Populate the paths in paths.sh to the corresponding location in your filesystem and run.

. paths.sh

to source them.

Transfer Methodologies.

We mainly studied the following methodologies from literature.

• CodeFormer
• VSAIT
• DECENT
• UNSB

For generating datasets and train different methodologies please clone the corresponding repositories.

RealDigiFace

This dataset is generated by simply passing the images of DigiFace1M to the methods that mentioned above, specifically using the CodeFormer and VSAIT that we report in the paper.

 DigiFace1M -> F -> RealDigiFace

In the third_party folder for each of the mentioned methods there exist a folder in which after cloning the original repository you need to override them with the content of these folder, These contains slight modifications of the original repository to generate the being able to translate the original images in the repository or measure the execution time.

Preparing Datasets for Training

Please follow the Dataset README.md.

Usually datasets for FR are presented in the following form.

Hierarchy structure (H1)

root_folder
|_id1___pic1.png
|  |____pic2.png
|       .
|       .
|  |____pic_id1_count.png
      .
      .
      .
|_idk___pic1.png
   |____pic2.png
       .
       .
   |____pic_idk_count.png

For example, using grid_generator.py you can generate sample grids of different variations of different identities.

  python3 grid_generator.py /path/to/digiface1m /path/to/realistic/version/of/digiface

General Scripts:

This is used to resize the images generated from different translation methods to the target resolution which usually used for FR.

• resize.py Given an folder which contains images in different hierarchy, it find and resize all the images into a destination folder while preserving the hierarchy of the folders.

    resize.py /path/to/source/folder /path/to/dest/folder --size 112 112 

• side_by_side.ipynb Notebook used to create the canvas of the pictures from the different methods. Like Fig.3 in the paper.

Face Recognition Experiments:

For face recognition experiments we used the modified version of the AdaFace repository. Please follow the instructions in this repository for training your models. With this repository we can train a FR system using the image folder hierarchy format in which folder corresponds to some samples of certain identities, and different identities are presented in separate folders.

Update

We also include the .rec format to train your models.

Scripts:

• generate_fr_confidence.ipynb: This notebook used to generate the tables for the IJB-C experiments, which given multiple experiments and their corresponding accuracies in different thresholds it aggregate and reports the mean and std of the accuracies.
• easy_benchmarks.ipynb: This notebook used to generate the tables from the benchmarks LFW, CFPFP, CALFW, ... from different experiments and their corresponding reported accuracies.
• transferred_vs_orig: Script use to study the effect of transfer in terms of the cosine similarity after and before applying realism.

Acknowledgment:

This project was funded under SAFER project from Hasler Foundation. We also thank the authors of the DigiFace1M, AdaFace, CodeFormer, VSAIT, DECENT, UNSB for sharing their amazing work.

Reference

If you find this repository useful for your work, please consider citing our paper:

@article{rahimi2024synthetic,
  title={Synthetic to Authentic: Transferring Realism to 3D Face Renderings for Boosting Face Recognition},
  author={Rahimi, Parsa and Razeghi, Behrooz and Marcel, Sebastien},
  journal={arXiv preprint arXiv:2407.07627},
  year={2024}
}