Skip to content
Snippets Groups Projects

[dask] Make vanilla-pad work properlly with dask

Merged Tiago de Freitas Pereira requested to merge fixes into master
All threads resolved!

In this MR, I'm.

  1. Wrapping the pipeline with DaskWrapper, because it wasn't. I think that's why things were not working for you with SGE @ydayer. vanilla-pad was creating SGE jobs AND running everything locally.
  2. Added the options "--dask-partition-size" and "--dask-n-workers", so the user has some freedom to set these parameters if the heuristics in place are unpleasant.

I'll merge this one right away because of the workshop.

Feel free to open issues if any

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Amir MOHAMMADI resolved all threads

    resolved all threads

  • mentioned in issue #37 (closed)

  • Please register or sign in to reply
    Loading