This class basically encodes the CLI command that bootstrap the worker
in a SGE job. Check here `https://distributed.dask.org/en/latest/resources.html#worker-resources` for more information
..note: This is class is temporary. It's basically a copy from SGEJob from dask_jobqueue.
The difference is that here I'm also handling the dask job resources tag (which is not handled anywhere). This has to be patched in the Job class. Please follow here `https://github.com/dask/dask-jobqueue/issues/378` to get news about this patch
client=Client(cluster)# start local workers as threads
returnclient
defsge_submit_to_idiap(spec):
"""
Submit a set of jobs to the Idiap cluster
Parameters
----------
spec: dict
A dictionary where the dict `key` is the SGE queue and the dict `value` contains another dictionary with the following elements that are self explained: `n_jobs`, `memory`, `sge_logs`, `io_big`, `resources`.
Returns
-------
A `dask.distributed.Client` containing all the SGE/dask workers submitted