Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in
bob.devtools
bob.devtools
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 5
    • Issues 5
    • List
    • Boards
    • Labels
    • Milestones
  • Merge Requests 1
    • Merge Requests 1
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • bob
  • bob.devtoolsbob.devtools
  • Issues
  • #53

Closed
Open
Opened May 18, 2020 by André Anjos@andre.anjos💬
  • Report abuse
  • New issue
Report abuse New issue

Implement hot-fix to repo indexing

To fix several issues that we are having with our conda channel, I am going to implement this hotfixing mechanism done for the defaults channel for our channel as well. This will:

  1. remove the need of moving of broken packages to our archive channel. Instead, we will keep the package in the same place but remove it from the index.
  2. allow us to fix broken packages in our channel. Like fixing bob.bio.base to make sure it does not get installed with numpy>=1.8.
  3. allow us to temporarily add packages (like mac versions of pytorch and torchvision) in our channel to fix our problems and remove them from channel index once the defaults channel catches up.

But to implement this care is needed from @bob users when they try to export an environment. In summary, you should avoid mixing conda env export and conda list --export --explicit. These 2 commands are designed in conda with two different goals and you should not use them for other purposes. I will explain what to do below:

Reproducibility and Repeatability of publications (bob.paper packages)

You should use conda list --export --explicit or even conda list --export --explicit --md5 to export your environment for other users to replicate your environment.

Save packages for future use:

    conda list --export --explicit > package-list.txt
or
    conda list --export --explicit --md5 > package-list.txt

Reinstall packages from an export file:

    conda create -n myenv --file package-list.txt

This method is, of course, not bullet proof but it should work reliably. If you use conda env export, the environment will most likely break in the future.

Share current ongoing work/projects (bob.project packages)

Sometimes, you want to share a common conda environment between colleagues while a project is continuing. You may even update this environment regularly. For this purpose, you should use conda env export or even better, create your environment.yml by hand. You may create environment files that work both on Linux and mac.

Create the environment file either by hand or by
    conda env export --file=environment.yml

Recreate the environment using:
    conda env create --file=environment.yml

Expect this environment to become broken from time to time and it might need updates. To avoid some breakage, do not pin the build strings, i.e. instead of bob.bio.base=4.1.0=py37h03d05df_0, write bob.bio.base=4.1.0. Also, you may want to only list your direct dependencies only.

Of course, you can choose to export to both formats in any scenario.

Edited May 18, 2020 by Amir MOHAMMADI
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
0
Labels
None
Assign labels
  • View project labels
Reference: bob/bob.devtools#53