Implement hot-fix to repo indexing
To fix several issues that we are having with our conda channel, I am going to implement this hotfixing mechanism done for the defaults channel for our channel as well. This will:
- remove the need of moving of broken packages to our archive channel. Instead, we will keep the package in the same place but remove it from the index.
- allow us to fix broken packages in our channel. Like fixing bob.bio.base to make
sure it does not get installed with
numpy>=1.8
. - allow us to temporarily add packages (like mac versions of pytorch and torchvision) in our channel to fix our problems and remove them from channel index once the defaults channel catches up.
But to implement this care is needed from @bob users when they try to export an environment.
In summary, you should avoid mixing conda env export
and conda list --export --explicit
.
These 2 commands are designed in conda with two different goals and you should not use them
for other purposes. I will explain what to do below:
Reproducibility and Repeatability of publications (bob.paper packages)
You should use conda list --export --explicit
or even conda list --export --explicit --md5
to export your environment for other users to replicate your environment.
Save packages for future use:
conda list --export --explicit > package-list.txt
or
conda list --export --explicit --md5 > package-list.txt
Reinstall packages from an export file:
conda create -n myenv --file package-list.txt
This method is, of course, not bullet proof but it should work reliably.
If you use conda env export
, the environment will most likely break in the future.
Share current ongoing work/projects (bob.project packages)
Sometimes, you want to share a common conda environment between colleagues while a
project is continuing. You may even update this environment regularly.
For this purpose, you should use conda env export
or even better,
create your environment.yml
by hand. You may create environment files that work
both on Linux and mac.
Create the environment file either by hand or by
conda env export --file=environment.yml
Recreate the environment using:
conda env create --file=environment.yml
Expect this environment to become broken from time to time and it might need updates.
To avoid some breakage, do not pin the build strings, i.e.
instead of bob.bio.base=4.1.0=py37h03d05df_0
, write bob.bio.base=4.1.0
.
Also, you may want to only list your direct dependencies only.
Of course, you can choose to export to both formats in any scenario.