As discussed previously, we need some verification protocols for this dataset.
Looking at the meds README (/ip/r***e/database/MEDS/), there are two things that we can analyse in this dataset, the ethnicity aspect (caucasian/black) and age. Unfortunatelly, we don't have enough data for gender.
Would be nice to have 3 fold verification protocol containing only men AND only black/caucasian such that:
worldhas all the samples that has only one image per identiy
devset has 50% of the images with more than one image per identiy
evalset has 50% of the images with more than one image per identiy
- for each fold you randomize the identities in the dev/eval set
Is it possible to carry this on?