Skip to content

Fixed multiqueue

Tiago de Freitas Pereira requested to merge multi into master

Hi @amohammadi @ydayer

I'm fixing here the issue raised with the multiqueue. I was wrongly setting all tasks to run in a particular resource restriction.

Now the problem is fixed.

To get it running you have to wrap your pipeline in the same way as before and fetch the resources like this

pipeline = bob.pipelines.wrap(
    ["sample", "checkpoint", "dask"],
    pipeline,
    model_path="./",
    transform_extra_arguments=(("metadata", "metadata"),),
    fit_tag="q_short_gpu",
)
from bob.pipelines.distributed.sge import get_resource_requirements
resources = get_resource_requirements(pipeline)

pipeline.fit_transform(X_as_sample).compute(
    scheduler=client, resources=resources
)

Merge request reports