This guide will explain how to develop existing |project| packages from their
source checkout. The sources of packages are hosted on Idiap's gitlab_ and they
are managed by git_. First we will explain how to setup a local environment,
later we will talk about how to checkout and build one or several packages from
their git_ source.
Very often, developers of |project| packages are confronted with the need to
clone repositories locally and develop installation/build and runtime code.
It is recommended to create isolated environments to develop new projects using conda_ and `zc.buildout`. Tools implemented in `bob.devtools` helps automate this process. In the following we talk about how to checkout and build one or several packages from
their git_ source and build proper isolated environments to develop them.
TLDR
TLDR
----
----
...
@@ -121,77 +119,117 @@ Optionally:
...
@@ -121,77 +119,117 @@ Optionally:
$ firefox sphinx/index.html # view the docs.
$ firefox sphinx/index.html # view the docs.
.. bob.extension.development_setup:
.. bob.devtools.local_development:
Local development environment
Local development of a Bob package
------------------------------
==================================
The first thing that you want to do is setup your local development
environment so you can start developing. Thanks to conda_ this is quite easy.
Here are the instructions:
* Install conda_ and learn about it a bit.
Checking out |project| package sources
* Add our `conda channel`_ to your channels.
--------------------------------------
|project| packages are developed through Gitlab_. In order to checkout a
Where ``<package>`` is the package you want to develop. Various packages exist
in |project|'s gitlab_ instance.
Create an isolated conda environment
------------------------------------
Now that we have the package checked out we need an isolated environment with proper configuration to develop the package. `bob.devtools` provides a tool that automatically creates such environment.
Before proceeding, you need to make sure that you already have a conda_ environment that has `bob.devtools` installed in. Refer to :ref:`bob.devtools.install` for information. Here we assume that you have a conda environment named `bdt` with installed `bob.devtools`
.. code-block:: sh
$ cd <package>
$ conda activate bdt
$ bdt create -vv dev
$ conda activate dev
Now you have an isolated conda environment with proper channels set. For more information about conda channels refer to `conda channel documentation`_.
.. note::
.. note::
Make sure you are using **only** our channel (with the highest priority)
When developing and testing new features, one often wishes to work against the very latest, *bleeding edge*, available set of changes on dependent packages.
and ``defaults`` (with the second highest priority). If you use any other
channel like ``conda-forge``, you may end-up with broken environments.
`bdt create` command adds `Bob beta channels`_ to highest priority which creates an environment with the latest dependencies instead of the latest *stable* versions of each package.
To see which channels you are using run:
To see which channels you are using run:
.. code-block:: sh
.. code-block:: sh
$ conda config --get channels
$ conda config --get channels
* Create a new environment that you will use for development.
.. note::
.. note::
We recommend creating a new conda_ environment for every project or task
We recommend creating a new conda_ environment for every project or task
that you work on. This way you can have several isolated development
that you work on. This way you can have several isolated development
environments which can be very different form each other.
environments which can be very different form each other.
Running buildout
----------------
The last step is to create a hooked-up environment so you can quickly test
local changes to your package w/o necessarily creating a conda-package. This
step is the easiest:
.. code-block:: sh
.. code-block:: sh
$ conda create --copy -n awesome-project \
$ cd <package> #if that is not the case
python=3 bob-devel
$ conda activate dev
$ source activate awesome-project
$ buildout
...
.. note::
zc.buildout_ works by modifying the load paths of scripts to find the correct
version of your package sources from the local checkout. After running,
buildout creates a directory called ``bin`` on your local package checkout. Use
the applications living there to develop your package. For example, if you need
to run the test suite:
The ``--copy`` option in the ``conda create`` command copies all files from
conda's cache folder into your environment. If you don't provide it, it
will try to create hard links to save up disk space but you will risk
modifying the files in **all** your environments without knowing. This can
easily happen if you use pip_ for example to manage your environment.
That's it. Now you have an **isolated** conda environment for your awesome
.. code-block:: sh
project! bob-devel_ is a conda meta package that pulls a set of common software
into your environment. To see what is installed, run:
$ ./bin/nosetests -sv
A python interpreter clone can be used to run interactive sessions:
.. code-block:: sh
.. code-block:: sh
$ conda list
$ ./bin/python
You can use conda_ and zc.buildout_ (which we will talk about later) to install
You can see what is installed in your environment:
some other libraries that you may require into your environment.
.. important::
.. code-block:: sh
Installing bob-devel_ **will not** install any |project| package. Use
$ conda list
conda_ to install the |project| packages that you require. For example to
get all the **core** `Bob packages`_ installed, run:
.. code-block:: sh
And you can install new necessary packages using conda:
.. code-block:: sh
$ conda install <package>
.. note::
If you want to debug a package regarding the issues showing on the ci you can use `bob.devtools`. Make sure the conda environment containing `bob.devtools` is activated.
$ cd <package>
$ conda activate bdt
$ bdt local build
$ conda install bob
One important advantage of using conda_ and zc.buildout_ is that it does
One important advantage of using conda_ and zc.buildout_ is that it does
**not** require administrator privileges for setting up any of the above.
**not** require administrator privileges for setting up any of the above.
...
@@ -199,7 +237,7 @@ Furthermore, you will be able to create distributable environments for each
...
@@ -199,7 +237,7 @@ Furthermore, you will be able to create distributable environments for each
project you have. This is a great way to release code for laboratory exercises
project you have. This is a great way to release code for laboratory exercises
or for a particular publication that depends on |project|.
or for a particular publication that depends on |project|.
.. _bob.extension.build_locally:
.. _bob.devtools.build_locally:
Building packages locally
Building packages locally
-------------------------
-------------------------
...
@@ -336,7 +374,7 @@ When the buildout command is invoked it will perform the following steps:
...
@@ -336,7 +374,7 @@ When the buildout command is invoked it will perform the following steps:
points in ``setup.py`` of a package, you need to run ``buildout`` again.
points in ``setup.py`` of a package, you need to run ``buildout`` again.
