[dask] Make vanilla-pad work properlly with dask (!81) · Merge requests · bob / bob.pad.base

Tiago de Freitas Pereira requested to merge fixes into master Dec 07, 2020

In this MR, I'm.

Wrapping the pipeline with DaskWrapper, because it wasn't. I think that's why things were not working for you with SGE @ydayer. vanilla-pad was creating SGE jobs AND running everything locally.
Added the options "--dask-partition-size" and "--dask-n-workers", so the user has some freedom to set these parameters if the heuristics in place are unpleasant.

I'll merge this one right away because of the workshop.

Feel free to open issues if any

[dask] Make vanilla-pad work properlly with dask