epoch checkpointing fix

fixed initialization of epochs when resuming training from a checkpoint

Thanks for this fix. I'm setting it to auto-merge once the pipeline succeeds.

enabled an automatic merge when the pipeline for 155715d5 succeeds

Unfortunately, these changes are not passing the quality check. Please read the logs and fix the issues. For the future, I recommend you check this before pushing. To do so, install pre-commit on your environment:

mamba install -c conda-forge pre-commit

Then, check all files:

pre-commit run --all-files

You may hook this to git, so the commit is not even accepted locally if the quality control does not pass. To hook it, just execute once:

pre-commit install

on the root of the repository to be affected.

I see, sorry for that. I should have done that before doing the request, saw it in the documentation too. I will fix it.

Also note that, when merging this request, you can attach issue #2 (closed), so it is closed when this MR is accepted (if this solves the issue). You can mention something like closes #2 of fixes #2 somewhere in your commit messages or the description of this MR to perform this "attachment" action.

aborted the automatic merge because source branch was updated

added 1 commit

• 458c855b - updated documentation

Compare with previous version

Something is not quite right with this MR - there 375+ changes for checkpointing fix?

Yes, I'll remove them.

I wanted to rebuild the documentation because I've also removed a parameter for the cli. I've run the command suggested in the idiap-devtools, "sphinx-build doc sphinx". It seems it was not intended to be used this way for this package, it created another seperate documentation folder that differs in name from the existing one, and I've pushed it without realizing it's not how the workflow seems to work.

added 1 commit

• 0fa5cb4b - removed duplicate documentation directory

Compare with previous version

@guelero: I added you to software/dev-profile> as a member. Can you please check you can trigger pipelines on this merge-request before I accept it? Thanks in advance!

Still the same error. I have no idea why it happens, is there something I can do?

I can access https://gitlab.idiap.ch/biosignal/software/dev-profile/ perfectly fine. And I can see the "developer" sign next to the project, too.

The problem is not biosignal/software/dev-profile>, but rather software/dev-profile>. Can you please check that one?

Oh. I don't have access to that one. Specifically, I get 404ed when accessing this link: https://gitlab.idiap.ch/software/dev-profile

I also still get 404ed when accessing this: https://gitlab.idiap.ch/biosignal/software/ptbench-ci-assets

Ah - you have a couple of different user names on gitlab, I think. This is the result of a first registration as an external Gitlab user, and then registering as an "Idiaper". I may have attributed access to the "wrong" you... Let me check this. Can you tell me which login you are using currently? Is it @guelero?

I have now uniformized it to exclusively use user @guelero. Please ensure not to use the login @ogueler anymore! Then, retry running a pipeline here.

Thank you, it starts now. Is it possible that I also need access to https://gitlab.idiap.ch/software/idiap-citools/' or https://gitlab.idiap.ch/software/idiap-citools ? It seems like the tests fail due to me not having access to it.

This package is used on all phases, some of which pass correctly. This may be a temporary issue with the whole site. Can you please try re-running one pipeline later?

Hummm - stratch that. Which of the two logins was given you after you have registered at Idiap? You may be using the wrong one...

OK, I think I understand it - you should be using user @ogueler instead of @guelero... The first one was added after your Idiap contract and makes you part of the "internal" group (with access to the "internal" projects, such as idiap-citools). Let me try to solve this in a more "permanent" way. This trial and guessing is a bit tough.

I see. I was using @guelero on gitlab until now. I tried to run the pipeline from @ogueler, but it seems like I cannot do that from that account either (a generic error message without any explanations), but I'm assuming this is due to this merge request being from @guelero.

Should I close this merge request, and open up a new merge request with @ogueler and see if that works? I will have to change some keys first on my machines.

OK - I added both @ogueler and @guelero as a developer to the following objects:

projects:

• software/dev-profile (you were a guest before, and I think this cannot work this way)
• software/idiap-citools

groups:

• biosignal/software

I hope this is OK now. Can you please retry running a pipeline?

added 2 commits

• 2381830d - re-added weight functionality + updated tests
• 010a0147 - gave precedence to checkpoints over weights

Compare with previous version

I changed the code to work the way it was intended to, as far as I can tell. I'm assuming it was never intended to load weights over an interrupted model when continuing training, so it cannot happen now.

EDIT: Should I remove or keep the weight loading feature for training? In deepdraw, it was not in there and I remember you instructing me to delete that part of the code, @andre.anjos

Right - please remove it and make it similar to biosignal/software/deepdraw>

I'll make this MR a Draft - please remove the "Draft" marker when you think it is ready for being merged.

marked this merge request as draft

added 1 commit

• 09e50ca4 - removed weight loading functionality

Compare with previous version

marked this merge request as ready

mentioned in commit 0209ebe1

merged