Block cancelling may result in NFS stale file problems
Upon the failure of one of the jobs in a block, it is possible that the scheduler ends-up erasing the temporary files being used by another job in the job set for the block. In an NFS-based filesystem, this will trigger an IO error in the iodaemon application. We must carefully study the synchronisation mechanism of the scheduler/worker/iodaemon to get into a workable solution for the future.
An experiment that triggers such problems: tpereira / tutorial / full_fisherface / 1 / PCA-200_LDA-49_periocular_mobio-male-AA1
Pay attention because the above experiment triggers this problem because of a programming error in another block. If that is fixed, it may not trigger it anymore. This is valid as of today, for beat.scheduler 0.5.3c1 and beat.core 0.11.3c1.