Caching does not work on docker based builds

Even though I get a message that cache is extracted, I end up downloading all the packages again each time:

Example: https://gitlab.idiap.ch/bob/bob.extension/-/jobs/91465

Blocks #35 (closed)