Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • beat.web beat.web
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 54
    • Issues 54
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 2
    • Merge requests 2
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • beatbeat
  • beat.webbeat.web
  • Issues
  • #325
Closed
Open
Issue created Nov 15, 2015 by André Anjos@andre.anjos💬Owner

[experiments] Reproducibility chart

After thinking about how reproducible an experiment in the platform really is, I think we can improve the experiment display a bit to include some sort of "reproducibility chart".

The idea behind this is to check, on a per-experiment basis and annotate, key points that may make the experiment irreproducible. Here are some key aspects

  1. The database used by the experiment is deactivated due to the end of the license agreement between Idiap and the controller
  2. The environment used by any of the blocks is not active anymore (outdated by another environment)
  3. There are incompatible API changes that may affect reproducibility (for example, the changes pushed on September 2nd. removing the data_index and data_index_end attributes). This condition must be taken with care, as it would require an (human) in-depth analysis of all experiment algorithms for a thorough conclusion. What we can do though, is to setup a new table that lists incompatible API changes and dates. If the experiment finished after that date, then we can consider it is compatible. Otherwise, not. We keep this table updated in case of eventual non-compatible API modifications.
Assignee
Assign to
Time tracking