Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • bob.pad.base bob.pad.base
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • bobbob
  • bob.pad.basebob.pad.base
  • Merge requests
  • !81

[dask] Make vanilla-pad work properlly with dask

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Tiago de Freitas Pereira requested to merge fixes into master Dec 07, 2020
  • Overview 1
  • Commits 1
  • Pipelines 1
  • Changes 1

In this MR, I'm.

  1. Wrapping the pipeline with DaskWrapper, because it wasn't. I think that's why things were not working for you with SGE @ydayer. vanilla-pad was creating SGE jobs AND running everything locally.
  2. Added the options "--dask-partition-size" and "--dask-n-workers", so the user has some freedom to set these parameters if the heuristics in place are unpleasant.

I'll merge this one right away because of the workshop.

Feel free to open issues if any

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: fixes