Skip to content
Snippets Groups Projects

[hotfix] [Dask] Add configuration to access multithreaded queue

Merged Laurent COLBOIS requested to merge sge-demanding into master

This adds an additional queue configuration that asks for multithreaded jobs, thus doubling the virtual memory limit to 16GB. With this fix I can run heavy Tensorflow baselines without OOM crash.

Long term it would be better to:

  • Expose the exact thread multiplier (now 2 by default)
  • Use the resource tagging of transformers to have the queue dynamically selected based on the transformer

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading