Skip to content
GitLab

[dask] Make vanilla-pad work properlly with dask

Merged Tiago de Freitas Pereira requested to merge fixes into master

In this MR, I'm.

  1. Wrapping the pipeline with DaskWrapper, because it wasn't. I think that's why things were not working for you with SGE @ydayer. vanilla-pad was creating SGE jobs AND running everything locally.
  2. Added the options "--dask-partition-size" and "--dask-n-workers", so the user has some freedom to set these parameters if the heuristics in place are unpleasant.

I'll merge this one right away because of the workshop.

Feel free to open issues if any