Expanding samples on the fly
I have a usecase where I want to expand one image to multiple images in the pipelinesimple
. Essentially, I can create
a transformer which can operate directly on samples, which takes in a SampleSet
with one image and return a SampleSet
with n
samples.
I managed to make it work, when every thing is in memory (with -m
option), and checkpointing is a problem since there are new samples. I can create new keys on the fly, what do you think is the best way to go about this.