Skip to content

Filelist datasets

André Anjos requested to merge datasets into master

@tiago.pereira, @ydayer: here is the CSV and JSON implementations of filelist-based datasets I had in my package, for your review.

The way the API goes makes sense for an application scenario in which loading an individual sample is costly (e.g. sample data is stored on disk).

If the sample data is not stored on disk (e.g., a dataset that can be completely stored in a single CSV table), then using other techniques would be better (e.g. pandas data frames).

I hope it is useful.

Merge request reports

Loading