Commit 82ae4eef authored by Amir MOHAMMADI's avatar Amir MOHAMMADI
Browse files

Remove explicit shutdown of dask clients in commands

parent f2dee741
Pipeline #44904 failed with stage
in 1 minute and 54 seconds
...@@ -51,7 +51,7 @@ It is possible to do it via configuration file ...@@ -51,7 +51,7 @@ It is possible to do it via configuration file
>>> database = .... # Biometric Database connector (class that implements the methods: `background_model_samples`, `references` and `probes`)" >>> database = .... # Biometric Database connector (class that implements the methods: `background_model_samples`, `references` and `probes`)"
\b \b
""" """
...@@ -193,8 +193,3 @@ def vanilla_biometrics( ...@@ -193,8 +193,3 @@ def vanilla_biometrics(
) )
logger.info("Experiment finished !!!!!") logger.info("Experiment finished !!!!!")
if dask_client is not None:
logger.info("Shutdown workers !!!!!")
dask_client.shutdown()
logger.info("Done !!!!!")
...@@ -47,7 +47,7 @@ $ bob pipelines vanilla-biometrics DATABASE PIPELINE -vv ...@@ -47,7 +47,7 @@ $ bob pipelines vanilla-biometrics DATABASE PIPELINE -vv
Check out all PIPELINE available by running: Check out all PIPELINE available by running:
`resource.py --types pipeline` `resource.py --types pipeline`
\b \b
and all available databases by running: and all available databases by running:
`resource.py --types database` `resource.py --types database`
...@@ -66,7 +66,7 @@ It is possible to do it via configuration file ...@@ -66,7 +66,7 @@ It is possible to do it via configuration file
>>> database = .... # Biometric Database connector (class that implements the methods: `background_model_samples`, `references` and `probes`)" >>> database = .... # Biometric Database connector (class that implements the methods: `background_model_samples`, `references` and `probes`)"
\b \b
""" """
...@@ -292,7 +292,7 @@ def vanilla_biometrics_ztnorm( ...@@ -292,7 +292,7 @@ def vanilla_biometrics_ztnorm(
def _build_filename(score_file_name, suffix): def _build_filename(score_file_name, suffix):
return os.path.join(score_file_name, suffix) return os.path.join(score_file_name, suffix)
# Running RAW_SCORES # Running RAW_SCORES
raw_scores = post_process_scores( raw_scores = post_process_scores(
pipeline, raw_scores, _build_filename(score_file_name, "raw_scores") pipeline, raw_scores, _build_filename(score_file_name, "raw_scores")
) )
...@@ -331,7 +331,3 @@ def vanilla_biometrics_ztnorm( ...@@ -331,7 +331,3 @@ def vanilla_biometrics_ztnorm(
_ = compute_scores(zt_normed_scores, dask_client) _ = compute_scores(zt_normed_scores, dask_client)
logger.info("Experiment finished !!!!!") logger.info("Experiment finished !!!!!")
Please register or sign in to reply
if dask_client is not None:
logger.info("Shutdown workers !!!!!")
dask_client.shutdown()
  • Hi @amohammadi,

    Why that? We should shutdown the client while using the CLI command.

  • dask already does this as part of their clean up.

  • It's just a good practice to clean up your stuff. You can have an exception in between and those workers can be opened for a while. It's better to leave this shutdown there.

  • I don't see any try and catch here. Anyway from what I have seen, as soon as the server crashes, dask clients automatically exit. This has been almost instant in my experience.

Please register or sign in to reply
logger.info("Done !!!!!")
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment