Hey @ydayer,

As discussed previously, we need some verification protocols for this dataset.

Looking at the meds README (/ip/r***e/database/MEDS/), there are two things that we can analyse in this dataset, the ethnicity aspect (caucasian/black) and age. Unfortunatelly, we don't have enough data for gender.

Would be nice to have 3 fold verification protocol containing only men AND only black/caucasian such that:

• the world has all the samples that has only one image per identiy
• the dev set has 50% of the images with more than one image per identiy
• the eval set has 50% of the images with more than one image per identiy
• for each fold you randomize the identities in the dev/eval set

Is it possible to carry this on?

Thanks

Cheers