Migrating the database interfaces to the new new CSV format

A new interface is being implemented in !300 (merged), and I am trying to migrate all dbs to that. This is a meta issue to track the migration.

• Convert all csv based ones to the new format
• Convert all custom interfaces to the new format
• Rewrite bob.bio.face.database file for database that were using custom interface
• Test and verify the new interfaces!
• Upload csv files according to https://gitlab.idiap.ch/bob/private/-/wikis/How-to-upload-resources
  • follow the data/bob/bob.bio.modality/bio-modality-database_name.tar.gz
  • Use this format in the code:

from bob.extension.download import get_file

name = "bio-modality-database_name-586b7e81.tar.gz"
dataset_protocols_path = get_file(
    name,
    # don't use https here, use http so the link works in the CI as well.
    [f"https://www.idiap.ch/software/bob/data/bob/bob.bio.modality/{name}",
     f"http://www.idiap.ch/software/bob/data/bob/bob.bio.modality/{name}"],
    cache_subdir="protocols",
    file_hash="586b7e81",
)
# remove .urls method.

Separate problems

• asvspoof2017-spoof the interface does not load
• utfvp has some columns without header in its CSV files!
• caspeal has some columns without header in its CSV files!
• gbu, rfw, lfw, ijbc, youtube interfaces are custom
• bob.bio.spear licit and spoof protocols need to change to new format like replaymobile

asvspoof2017-spoof the interface does not load

changed the description

changed the description

changed the description

assigned to @lcolbois

ping @ydayer @flavio.tarsetti This is the overarching issue for the CSV Dataset conversion.

Linked to this are:

• The merge request with the new CSV Dataset interface !300 (merged)
• The "god" script to convert already implemented CSV dataset to the new format https://gitlab.idiap.ch/bob/database-purgatory/-/merge_requests/1/commits

As discussed with Flavio today:

• While this is not necessarily urgent (it is definitely a good improvement of the API but the current version holds although it is clunky), the more we wait the more we risk to have more CSV implementations to port.
• When this is over, it might be good to raise a warning (in the code and/or the doc) for users, asking them to double check the implementation when they use a specific database implementation for the first time after the porting. This warning could then be removed once the database has been successfully used in some project.

I'll be in vacation next week so not in Bob meeting, but we can have a chat tomorrow about this if you have the time.

Focus first on bob.bio.face

• Generate score files and metrics for 1 specific baseline and all altered datasets, in Bob 11.1
• Generate CSV files using Amir's script
• Upload on WebDAV (DO NOT ERASE OLD VERSIONS YET, try naming unformization.
• Rewrite the database interface in bob.bio.base
  • Mostly path and hash change for files that are already CSV
  • Rewrite interface for custom databases
• Rerun same baseline, with new CSV files / new implementation

I will run experiments with all the bob.bio.face datasets (from Bob 11.1) with the iresnet34 model, to establish reference scores before updating the CSV API and the CSV files.

I ran the experiments, however some of them failed:

• ATNT
• Casia-Africa
• All Caspeal protocols
• ijbc-test4-g1
• replaymobile-img

Didn't investigate yet the reason for those failures. Technically I guess it would be rather pointless to fix their current implementation just for the sake of it, so maybe we can first convert them to the CSV format then check that we can run them ?

Only drawback is that we cannot rely on the scores to ensure the conversion was correct... but I don't even know since how long they are broken.

Should be OK now. Eventual problems will be raised in new issues.

closed