Skip to content
Snippets Groups Projects

Fixing the temp directory path for legacy algorithms IF checkpointing is off

Merged Tiago de Freitas Pereira requested to merge fix-legacy into master

Hi @lcolbois,

Thanks for detecting the problem and opening the issue with all the details.

The problem is that most of the legacy algorithms are not pickled serializable, therefore, we can't wrap their transformations as :any:bob.pipelines.Sample and use pipelines in memory (vanilla-biometrics without the --checkpoint option). Hence, we need to make them automatically "checkpointable" and use their legacy functions to persist data into the disk in a temporary directory.

This was wrongly set in the lgbphs baseline. The /tmp/ was being set by default and this is not valid while running stuff on cluster. Basically, you are persisting data on the /tmp of every node and obviously, this directory is not visible in other nodes. That's why things work locally and on the grid with checkpoints.

In this MR bob.bio.base!238 (merged) I formalized a function that generates proper temporary directories that are visible on the cluster.

closes #39 (closed)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading