Add custom Tensoroard logger, remove CSV logger

The default behavior of Lightning's loggers is problematic, as they create a new file each time an experiment is run and it is not possible to resume logging using an existing file.

Each log file is in a subfolder named version_n. Specifying a version when initializing the logger will overwrite that log instead of appending to it.

Lightning allows for the creation of custom loggers. We should implement our own version of TensorBoardLogger which can save logs without version numbering and can resume logging if a file already exists.

In addition, we could remove the CSV logger and implement a translator from TensorBoard logs into CSV instead.

Implementation of the TensorBoardLogger. It inherits from the Fabric library.

• https://lightning.ai/docs/pytorch/stable/_modules/lightning/pytorch/loggers/tensorboard.html#TensorBoardLogger
• https://lightning.ai/docs/fabric/latest/_modules/lightning/fabric/loggers/tensorboard.html#TensorBoardLogger

Our custom logger could probably be simplified. This explains what a minimal logger implementation looks like:

https://lightning.ai/docs/pytorch/stable/extensions/logging.html#make-a-custom-logger

assigned to @dcarron

Please cross-check #10 (closed) while you're there!

mentioned in issue #10 (closed)

I removed the CSVLogger on one of the previous commits at 026ddb48. If we can somehow merge the logs when we continue from the same log directory, we would be good to close this one.

I was able to change the location where logs are saved simply by redefining the log_dir() property.

So now all logs are grouped inside the same directory, for example results/logs. There is no version_n anymore.

It will still create one file per run, but tensorboard will combine them together when displaying the graphs. See the following screenshots, where the logs contain 4 files for a total of 13 epochs.

It does not seem possible to specify the name of the logfiles, as that is generated at runtime by tensorflow: https://github.com/tensorflow/tensorboard/blob/2a7438801cd307a1893a5e729c4c5f392275d637/tensorboard/summary/writer/event_file_writer.py#L73

mentioned in commit 24079fc2

This looks great already. I closed this issue with the commit above, with a minor logger fix after these changes. We may move on, thanks.

mentioned in commit b2ff0f0d

mentioned in commit adf29dd1

closed with merge request !6 (merged)