Fixed multiqueue
Compare changes
I'm fixing here the issue raised with the multiqueue. I was wrongly setting all tasks to run in a particular resource restriction.
Now the problem is fixed.
To get it running you have to wrap your pipeline in the same way as before and fetch the resources like this
pipeline = bob.pipelines.wrap(
["sample", "checkpoint", "dask"],
pipeline,
model_path="./",
transform_extra_arguments=(("metadata", "metadata"),),
fit_tag="q_short_gpu",
)
from bob.pipelines.distributed.sge import get_resource_requirements
resources = get_resource_requirements(pipeline)
pipeline.fit_transform(X_as_sample).compute(
scheduler=client, resources=resources
)