diff --git a/templates/README.md b/templates/README.md
index 52e1226a5ba95b0a31905a1ef6746f7c4c3b7984..06f437d092d6e27ab77b32dd47ccbd5f12b5f9b4 100644
--- a/templates/README.md
+++ b/templates/README.md
@@ -6,11 +6,14 @@ touched by this procedure. You **don't** have to do all these steps at once. If
you choose to only do some, prioritise by the order in this text (first do the
continuous integration changes, then the licensing checks, etc).
-> **Notice**: This text may change as we get more experience in what needs to be changed. We may also automatise some of the actions below. We'll keep you posted in this case and update these instructions.
+> **Notice**: This text may change as we get more experience in what needs to
+> be changed. We may also automatise some of the actions below. We'll keep you
+> posted in this case and update these instructions.
## 0. Set the right origin
-If you haven't checked your repository from gitlab yet, you don't have to remove it and re-clone it, you can just set its origin to point to gitlab:
+If you haven't checked your repository from gitlab yet, you don't have to
+remove it and re-clone it, you can just set its origin to point to gitlab:
```sh
$ git remote set-url origin git@gitlab.idiap.ch:bob/`basename $(pwd)`
@@ -20,12 +23,6 @@ $ git pull
## 1. Continuous Integration
-Remove the `.travis.yml` file from your package:
-
-```sh
-$ git rm -f .travis.yml
-```
-
Copy the stock yml template for the CI builds:
@@ -50,19 +47,220 @@ You also need to enable the following options on your project:
facedemo`) are enabled.
3. Visit the "Runners" section of your package settings and enable all runners
with the `docker` and `macosx` tags.
-4. Go into the "Variables" section of your package setup and **add common
- variables** corresponding to the usernames and passwords for uploading
- wheels and documentation tar balls to our (web DAV) server, as well as PyPI
- packages. You can copy required values from [the "Variables" section of
- bob.admin](https://gitlab.idiap.ch/bob/bob.admin/variables). N.B.: You
- **must** be logged into gitlab to access that page.
-5. Make sure to **disable** the service "Build e-mails" (those are very
- annoying)
-6. Setup the coverage regular expression under "CI/CD pipelines" to have the
+4. Setup the coverage regular expression under "CI/CD pipelines" to have the
value `^TOTAL.*\s+(\d+\%)$`, which is adequate for figuring out the output
of `coverage report`
+## 1.5 Conda recipe
+
+The CI system is based on conda recipes to build the package. The recipes must
+be located in the `conda` folder of each package. You can start modifying the
+recipe of each package by copying the template recipe:
+
+```sh
+$ mkdir -p conda
+$ curl -k --silent https://gitlab.idiap.ch/bob/bob.admin/raw/master/templates/meta.yaml > conda/meta.yaml
+$ sed -i "s/<PACKAGE>/`basename $(pwd)`/g" conda/meta.yaml
+```
+
+You should refrain from modifying the recipe except for the places that you are
+asked to modify. We want to keep recipes as similar as possible so that
+updating all of them in future would be possible by a script.
+
+Each recipe is unique to the package and need to be further modified by the
+package maintainer to work. The reference definition of the `meta.yaml` file
+is here:
+https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html
+The `meta.yaml` file (referred to as the recipe) will contain
+duplicate information that is already documented in `setup.py`,
+`requirements.txt`, and `test-requirements.txt`. For the time being you have to
+maintain both the `meta.yaml` file and the other
+files. We have plans to remove this duplication in future:
+https://gitlab.idiap.ch/bob/bob.extension/issues/38
+You are welcome to contribute to that.
+
+Let's walk through the `meta.yaml` file (the recipe) that you just have
+downloaded and further customize it to our package. You need to carry out all
+the steps below otherwise the template `meta.yaml` is not usable as it is.
+
+### ENTRY_POINTS
+
+You need to check if your package has any `console_scripts`. These are
+documented in `setup.py` of each package. You need to list the
+`console_scripts` entry points (only `console_scripts`; other entry points
+**should not** be listed in `meta.yaml`.) in the build section of the
+recipe.
+
+* If there are no `console_scripts`, remove the line (line 9) that says:
+ `<ENTRY_POINTS>`.
+* If there are some, list them in the `meta.yaml` file as well:
+ https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#python-entry-points
+ For example, if the `setup.py` file contains:
+
+ ```python
+ entry_points={
+ 'console_scripts': [
+ 'jman = gridtk.script.jman:main',
+ 'jgen = gridtk.script.jgen:main',
+ ]
+ ```
+ You would replace `<ENTRY_POINTS>` with:
+
+ ```yaml
+ build: # this line is already in the recipe. Do not add. Tabbing (two sapces) is important.
+ entry_points:
+ - jman = gridtk.script.jman:main
+ - jgen = gridtk.script.jgen:main
+ ```
+
+### BUILD_DEPS and HOST_DEPS in requirements.
+
+This part of the recipe lists the packages that are required during build time:
+https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#requirements-section
+Having build and host requirements separately enables cross-compiling of the recipes.
+
+* If the packages does not contain C/C++ code, remove the line that says (line 21) `<BUILD_DEPS>`
+* Otherwise, you need to replace `<BUILD_DEPS>` with:
+ ```yaml
+ requirements: # this line is already in the recipe. Do not add. Tabbing (two sapces) is important.
+ build:
+ - {{ compiler('c') }}
+ - {{ compiler('cxx') }}
+ - pkg-config {{ pkg_config }}
+ - cmake {{ cmake }}
+ ```
+ The pkg-config and cmake lines are optional. If the package uses them, you need to list them.
+
+* List all the packages that are in `requirements.txt` in the
+ `requirements:host` section by removing `<HOST_DEPS>` and adding a new line
+ per dependency. For example, here is what `bob.measure` has in its host:
+ ```yaml
+ host: # this line is already in the recipe. Do not add. Tabbing (two sapces) is important.
+ - python {{ python }} # this line is already in the recipe. Do not add.
+ - setuptools {{ setuptools }} # this line is already in the recipe. Do not add.
+ - bob.extension
+ - bob.blitz
+ - bob.core
+ - bob.math
+ - bob.io.base
+ - matplotlib {{ matplotlib }}
+ - libblitz {{ libblitz }}
+ - boost {{ boost }}
+ - numpy {{ numpy }}
+ - docopt {{ docopt }}
+ ```
+ You need to add a jinja variable like `{{ dependency }}` in front of the
+ dependencies that we *do not* develop. The jinja variable name should not
+ contain `.` or `-`; replace those with `_`. The bob packages (and gridtk)
+ should be listed as is.
+
+* Unlike `pip`, `conda` is not limited to Python programs. If the package
+ depends on some non-python package (like `boost`), you need to list it in the
+ `host` section.
+
+### RUN_DEPS
+
+In the `requirements:run` section, you will list dependencies that are needed
+when a package is used (run-time) dependencies. Usually, for pure-Python
+packages you list the same packages as in the host section also in the run
+section. This is simple **BUT** conda build 3 introduced a new concept named
+`run_exports`
+https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#pin-downstream
+which makes this complicated. In summary, you put
+all the run-time dependencies in the `requirements:run` section **UNLESS** this
+dependency was listed in the host section **and** the dependency has a
+`run_exports`. The problem is that you cannot easily find which packages
+actually do have `run_exports` unless you look at their conda recipe. Usually,
+all the C/C++ libraries like `jpeg`, `hdf5` have `run_exports` (`boost` does
+not have one for example). All `bob` packages have this too. For example, here
+is what is inside the `requirements:run` section of `bob.measure`:
+```yaml
+ run: # this line is already in the recipe. Do not add.
+ - python # this line is already in the recipe. Do not add.
+ - setuptools # this line is already in the recipe. Do not add.
+ - matplotlib
+ - boost
+ - {{ pin_compatible('numpy') }}
+ - docopt
+```
+The `pin_compatible` jinja function is explained in
+https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#pin-downstream
+You need to use it on `numpy` if and only if you use `numpy` in C level.
+Otherwise, just list numpy normally. We do not know of any other package
+besides numpy used in C level that needs to use the `pin_compatible` jinja
+function.
+
+Here is a list of packages that we know that they have `run_exports`:
+```
+- bzip2
+- dbus
+- expat
+- ffmpeg
+- fontconfig
+- freetype
+- giflib
+- glib
+- gmp
+- gst-plugins-base
+- gstreamer
+- hdf5
+- icu
+- jpeg
+- kaldi
+- libblitz
+- libboost
+- libffi
+- libmatio
+- libogg
+- libopus
+- libpng
+- libsvm
+- libtiff
+- libvpx
+- libxcb
+- libxml2
+- menpo
+- mkl # not this one but mkl-devel has so no need to list mkl if you use mkl-devel in host
+- mkl-devel
+- ncurses
+- openfst
+- openssl
+- readline
+- sox
+- speex
+- speexdsp
+- sqlite
+- tk
+- vlfeat
+- xz
+- yaml
+- zlib
+```
+
+### ENTRY_POINTS_TEST
+
+If you listed some console_sripts in the `build:entry_points` section, you will
+test them here. Otherwise, remove the line that says `<ENTRY_POINTS_TEST>`. For
+example, if you had the examples entry points above, you would test them like:
+```yaml
+test:
+ imports:
+ - {{ name }}
+ commands:
+ - jman --help
+ - jgen --help
+```
+
+### TEST_DEPS
+
+https://conda.io/docs/user-guide/tasks/build-packages/define-metadata.html#test-requirements
+You need to list the packages here that are reuired during test-time only. By
+default, we have `bob-devel`, `nose`, `coverage`, `sphinx`, ... do not remove
+them. The test-time dependencies are listed in `test-requirements.txt`. If this
+file does not exist, just remove the line that says `<TEST_DEPS>`.
+
+
## 2. Licensing
Verify if the license of your package satisfies what is written on our
@@ -235,7 +433,3 @@ text files to be placed along `conf.py` (on the same directory):
## 7. Update the logo on your project
Please update the logo of your project (on the Settings), just set it to [this one](templates/bob-128x128.png).
-
-## 8. Create a conda recipe for the package
-
-Please create a conda build recipte for the new package and put it in bob.conda.