Skip to content
Snippets Groups Projects
Commit 07c0cad1 authored by André Anjos's avatar André Anjos :speech_balloon:
Browse files

Documentation updates

parent 6058ac76
No related branches found
No related tags found
1 merge request!280Documentation updates; Buildout and Nose deprecation
Pipeline #59508 canceled
Stage: build
Hide whitespace changes
Inline Side-by-side
README.rst
+ 6
− 6
View file @ 07c0cad1
.. vim: set fileencoding=utf-8 : .. vim: set fileencoding=utf-8 :
.. image:: https://img.shields.io/badge/docs-v4.2.0-orange.svg .. image:: https://img.shields.io/badge/docs-latest-orange.svg
:target: https://www.idiap.ch/software/bob/docs/bob/bob.devtools/v4.2.0/index.html :target: https://www.idiap.ch/software/bob/docs/bob/bob.devtools/master/index.html
.. image:: https://gitlab.idiap.ch/bob/bob.devtools/badges/v4.2.0/pipeline.svg .. image:: https://gitlab.idiap.ch/bob/bob.devtools/badges/master/pipeline.svg
:target: https://gitlab.idiap.ch/bob/bob.devtools/commits/v4.2.0 :target: https://gitlab.idiap.ch/bob/bob.devtools/commits/master
.. image:: https://gitlab.idiap.ch/bob/bob.devtools/badges/v4.2.0/coverage.svg .. image:: https://gitlab.idiap.ch/bob/bob.devtools/badges/master/coverage.svg
:target: https://gitlab.idiap.ch/bob/bob.devtools/commits/v4.2.0 :target: https://gitlab.idiap.ch/bob/bob.devtools/commits/master
.. image:: https://img.shields.io/badge/gitlab-project-0000c0.svg .. image:: https://img.shields.io/badge/gitlab-project-0000c0.svg
:target: https://gitlab.idiap.ch/bob/bob.devtools :target: https://gitlab.idiap.ch/bob/bob.devtools
......
doc/development.rst
+ 63
− 46
View file @ 07c0cad1
...@@ -4,23 +4,23 @@ ...@@ -4,23 +4,23 @@
Local development of packages Local development of packages
=============================== ===============================
Very often, developers are confronted with the need to Very often, developers are confronted with the need to clone package
clone package repositories locally and develop installation/build and runtime code. repositories locally and develop installation/build and runtime code. It is
It is recommended to create isolated environments using conda_ to develop new projects. recommended to create isolated environments using conda_ to develop new
Tools implemented in ``bob.devtools`` helps automate this process for |project| packages. projects. Tools implemented in ``bob.devtools`` help automate this process for
In the following we talk about how to checkout and build one or several packages |project| packages. In the following we talk about how to checkout and build
from their git_ source and build proper isolated environments to develop them. one or several packages from their git_ source and build proper isolated
Then we will describe how to create a new bob package from scratch and develop environments to develop them. Then we will describe how to create a new bob
existing bob packages along side it. package from scratch and develop existing bob packages along side it.
TLDR TL;DR
==== =====
Suppose you want to checkout the package ``bob.blitz`` from source and start developing it locally. Suppose you want to checkout the package ``bob.blitz`` from source and start
We will use the tools implemented in ``bob.devtools`` to create a proper developing it locally. We will use the tools implemented in ``bob.devtools`` to
developing environment to build and develop ``bob.blitz``. create a proper developing environment to build and develop ``bob.blitz``. We
We assume you have ``bob.devtools`` installed on a conda environment named assume you have ``bob.devtools`` installed on a conda environment named ``bdt``
``bdt`` (Refer to :ref:`bob.devtools.install` for detailed information.) (Refer to :ref:`bob.devtools.install` for detailed information.)
* Checkout the source of the package from git: * Checkout the source of the package from git:
...@@ -69,7 +69,7 @@ for example: ...@@ -69,7 +69,7 @@ for example:
.. code-block:: sh .. code-block:: sh
$ nosetests -sv $ pytest -sv ...
* Some packages may come with a pre-commit_ config file (``.pre-commit-config.yaml``). * Some packages may come with a pre-commit_ config file (``.pre-commit-config.yaml``).
Make sure to install pre-commit if the config file exists: Make sure to install pre-commit if the config file exists:
...@@ -81,19 +81,23 @@ for example: ...@@ -81,19 +81,23 @@ for example:
$ pip install pre-commit $ pip install pre-commit
$ pre-commit install $ pre-commit install
.. bob.devtools.local_development: .. bob.devtools.local_development:
Local development of existing packages Local development of existing packages
====================================== ======================================
To develop existing |project| packages you need to checkout their source code and install them in your environment.
To develop existing |project| packages you need to checkout their source code
and install them in your environment.
Checking out package sources Checking out package sources
---------------------------- ----------------------------
|project| packages are developed through gitlab_. Various packages exist in |project| packages are developed through gitlab_. Various packages exist in
|project|'s gitlab_ instance. In the following we assume you want to install and |project|'s gitlab_ instance. In the following we assume you want to install
build locally the ``bob.blitz`` package. In order to checkout a package, just and build locally the ``bob.blitz`` package. In order to checkout a package,
use git_: just use git_:
.. code-block:: sh .. code-block:: sh
...@@ -104,8 +108,13 @@ use git_: ...@@ -104,8 +108,13 @@ use git_:
Create an isolated conda environment Create an isolated conda environment
------------------------------------ ------------------------------------
Now that we have the package checked out we need an isolated environment with proper configuration to develop the package. ``bob.devtools`` provides a tool that automatically creates such environment. Now that we have the package checked out we need an isolated environment with
Before proceeding, you need to make sure that you already have a conda_ environment with ``bob.devtools`` installed in it (Refer to :ref:`bob.devtools.install` for more information). let's assume that you have a conda environment named ``bdt`` with installed ``bob.devtools``. proper configuration to develop the package. ``bob.devtools`` provides a tool
that automatically creates such environment. Before proceeding, you need to
make sure that you already have a conda_ environment with ``bob.devtools``
installed in it (Refer to :ref:`bob.devtools.install` for more information).
let's assume that you have a conda environment named ``bdt`` with installed
``bob.devtools``.
.. code-block:: sh .. code-block:: sh
...@@ -114,22 +123,27 @@ Before proceeding, you need to make sure that you already have a conda_ environm ...@@ -114,22 +123,27 @@ Before proceeding, you need to make sure that you already have a conda_ environm
$ conda activate dev $ conda activate dev
Now you have an isolated conda environment named `dev` with proper channels set. Now you have an isolated conda environment named `dev` with proper channels
For more information about conda channels refer to `conda channel set. For more information about conda channels refer to `conda channel
documentation`_. documentation`_.
The `bdt dev create` command assumes a directory named `conda`, exists on the The ``bdt dev create`` command assumes a directory named ``conda``, exists on
current directory. The `conda` directory contains a file named `meta.yaml`, that the current directory. The ``conda`` directory contains a file named
is the recipe required to create a development environment for the package you ``meta.yaml``, that is the recipe required to create a development environment
want to develop. for the package you want to develop.
.. note:: .. note::
When developing and testing new features, one often wishes to work against the very latest, *bleeding edge*, available set of changes on dependent packages. When developing and testing new features, one often wishes to work against
the very latest, *bleeding edge*, available set of changes on dependent
packages.
`bdt dev create` command adds `Bob beta channels`_ to highest priority which creates an environment with the latest dependencies instead of the latest *stable* versions of each package. ``bdt dev create`` command adds `Bob beta channels`_ to highest priority
which creates an environment with the latest dependencies instead of the
latest *stable* versions of each package.
If you want to create your environment using *stable* channels, you can use this command instead: If you want to create your environment using *stable* channels, you can use
this command instead:
.. code-block:: sh .. code-block:: sh
...@@ -142,7 +156,6 @@ want to develop. ...@@ -142,7 +156,6 @@ want to develop.
$ conda config --get channels $ conda config --get channels
.. note:: .. note::
We recommend creating a new conda_ environment for every project or task We recommend creating a new conda_ environment for every project or task
...@@ -165,14 +178,14 @@ To run the test suite: ...@@ -165,14 +178,14 @@ To run the test suite:
.. code-block:: sh .. code-block:: sh
$ nosetests -sv $ pytest -sv ...
or build the documentation: or build the documentation:
.. code-block:: sh .. code-block:: sh
$ sphinx-build -aEn doc sphinx # make sure it finishes without warnings. $ sphinx-build -aEn doc sphinx # make sure it finishes without warnings.
$ firefox sphinx/index.html # view the docs. $ xdg-open sphinx/index.html # view the docs.
You can see what is installed in your environment: You can see what is installed in your environment:
...@@ -189,7 +202,9 @@ And you can install new packages using conda: ...@@ -189,7 +202,9 @@ And you can install new packages using conda:
.. note:: .. note::
If you want to debug a package regarding an issues showing on the ci you can use `bob.devtools`. Make sure the conda environment containing `bob.devtools` is activated. If you want to debug a package regarding an issues showing on the ci you
can use ``bob.devtools``. Make sure the conda environment containing
``bob.devtools`` is activated (typically, ``base``).
.. code-block:: sh .. code-block:: sh
...@@ -203,14 +218,16 @@ be able to create distributable environments for each project you have. This is ...@@ -203,14 +218,16 @@ be able to create distributable environments for each project you have. This is
a great way to release code for laboratory exercises or for a particular a great way to release code for laboratory exercises or for a particular
publication that depends on |project|. publication that depends on |project|.
Developing multiple existing packages simultaneously Developing multiple existing packages simultaneously
---------------------------------------------------- ----------------------------------------------------
It so happens that you want to develop several packages against each other for It so happens that you want to develop several packages against each other for
your project. Let's assume you want to develop ``bob.blitz`` and your project. Let's assume you want to develop ``bob.blitz`` and
``bob.extension`` simultaneously. ``bob.blitz`` is dependent on ``bob.extension`` simultaneously. ``bob.blitz`` is dependent on
``bob.extension``. First we checkout package ``bob.blitz`` and build an isolated ``bob.extension``. First we checkout package ``bob.blitz`` and build an
conda environment as explained in the previous section. Then checkout and isolated conda environment as explained in the previous section. Then checkout
install ``bob.extension`` as following: and install ``bob.extension`` as following:
.. code-block:: sh .. code-block:: sh
...@@ -262,14 +279,14 @@ You need to checkout and install these packages: ...@@ -262,14 +279,14 @@ You need to checkout and install these packages:
$ bdt dev install src/bob.extension src/bob.blitz # the order of installing dependencies matters! $ bdt dev install src/bob.extension src/bob.blitz # the order of installing dependencies matters!
When you build your new package, it is customary to checkout the dependent When you build your new package, it is customary to checkout the dependent
packages (in this example ``bob.extension`` and ``bob.blitz``) in the `src` packages (in this example ``bob.extension`` and ``bob.blitz``) in the ``src``
folder in the root of your project. folder in the root of your project.
As usual, first create an isolated conda environment using `bdt dev create` command. As usual, first create an isolated conda environment using ``bdt dev create``
Some of bob packages need dependencies that might not be installed on your command. Some of bob packages need dependencies that might not be installed on
environment. You can find these dependencies by checking `conda/meta.yaml` of your environment. You can find these dependencies by checking
each package. Install the required packages and then run `bdt dev install`. For ``conda/meta.yaml`` of each package. Install the required packages and then run
our example you need to do the following: ``bdt dev install``. For our example you need to do the following:
.. code-block:: sh .. code-block:: sh
......
doc/install.rst
+ 23
− 14
View file @ 07c0cad1
...@@ -12,22 +12,32 @@ channels: ...@@ -12,22 +12,32 @@ channels:
.. code-block:: sh .. code-block:: sh
$ conda create -n bdt -c https://www.idiap.ch/software/bob/conda bob.devtools $ conda install -n base -c https://www.idiap.ch/software/bob/conda bob.devtools
# or, for beta releases: # or, for beta releases:
$ conda create -n bdt -c https://www.idiap.ch/software/bob/conda/label/beta -c https://www.idiap.ch/software/bob/conda bob.devtools $ conda install -n base -c https://www.idiap.ch/software/bob/conda/label/beta -c https://www.idiap.ch/software/bob/conda bob.devtools
If you use one of our supported Python versions on your base environment, you .. warning::
may also install ``bdt`` on it (recommended):
.. code-block:: sh Some commands from this package will use the ``conda`` CLI to install
packages on new environments.
$ conda install -n base -c https://www.idiap.ch/software/bob/conda bob.devtools If you install bob.devtools on another environment which is not ``base``, a
# or, for beta releases: new conda package-cache will be created on that environment, possibly
$ conda install -n base -c https://www.idiap.ch/software/bob/conda/label/beta -c https://www.idiap.ch/software/bob/conda bob.devtools duplicating the size of your conda installation. For this reason, we
recommend you install this package on the ``base`` environment.
We provide packages for both 64-bit Linux and MacOS, for Python 3.6+. Once We provide packages for both 64-bit Linux and MacOS, for Python 3.8+. Once
installed, you can use these tools within the created environment like this: installed, you can use these tools within the created environment like this:
.. code-block:: sh
$ conda activate base
(base) $ bdt --help
You may also hook ``bdt`` on your global ``PATH`` variable, and avoid having to
activate ``base`` to use the command like this:
.. code-block:: sh .. code-block:: sh
$ ln -s $(which bdt) ~/.local/bin $ ln -s $(which bdt) ~/.local/bin
...@@ -89,11 +99,10 @@ pip-installable packages, create a section named ``create`` in the file ...@@ -89,11 +99,10 @@ pip-installable packages, create a section named ``create`` in the file
[create] [create]
pip_extras = ipdb pip_extras = ipdb
bob.buildout pre-commit
mr.developer
Then, by default, ``bdt dev create`` will automatically pip install ``ipdb`` and Then, by default, ``bdt dev create`` will automatically pip install ``ipdb``
``mr.developer`` at environment creation time. You may reset this list to your and ``pre-commit`` at environment creation time. You may reset this list to
liking. your liking.
.. include:: links.rst .. include:: links.rst
doc/links.rst
+ 1
− 1
View file @ 07c0cad1
...@@ -12,7 +12,7 @@ ...@@ -12,7 +12,7 @@
.. _Bob beta channels: https://www.idiap.ch/software/bob/conda/label/beta .. _Bob beta channels: https://www.idiap.ch/software/bob/conda/label/beta
.. _git: http://git-scm.com/ .. _git: http://git-scm.com/
.. _gitlab: https://gitlab.idiap.ch/bob/ .. _gitlab: https://gitlab.idiap.ch/bob/
.. _nose: https://nose.readthedocs.org/en/latest/ .. _pytest: https://docs.pytest.org/
.. _python: http://www.python.org .. _python: http://www.python.org
.. _pypi: http://pypi.python.org .. _pypi: http://pypi.python.org
.. _setuptools: https://setuptools.readthedocs.io .. _setuptools: https://setuptools.readthedocs.io
......
doc/release.rst
+ 2
− 2
View file @ 07c0cad1
...@@ -86,8 +86,8 @@ Here are the instructions to release Bob meta package: ...@@ -86,8 +86,8 @@ Here are the instructions to release Bob meta package:
- bob.ip.binseg ==1.1.0 # [linux] - bob.ip.binseg ==1.1.0 # [linux]
* Test the conda recipe of bob. You may want to cancel the * Test the conda recipe of bob. You may want to cancel the command below once
command below once it reaches the nosetests.: it reaches the unit tests:
.. code-block:: sh .. code-block:: sh
......
doc/templates.rst
+ 9
− 12
View file @ 07c0cad1
...@@ -25,26 +25,23 @@ These instructions describes some steps that needs to be noted after creating ne ...@@ -25,26 +25,23 @@ These instructions describes some steps that needs to be noted after creating ne
Unit tests Unit tests
========== ==========
Writing unit tests is an important asset on code that needs to run in different platforms and a great way to make sure all is OK. Writing unit tests is an important asset on code that needs to run in different
Test units are run with nose_. platforms and a great way to make sure all is OK. Test units are run with
To run the test units on your package call: pytest_. To run the test units on your package call:
.. code-block:: sh .. code-block:: sh
$ nosetests -v $ pytest -sv ...
bob.example.library.test.test_reverse ... ok bob.example.library.test.test_reverse ... ok
----------------------------------------------------------------------
Ran 1 test in 0.253s
OK This example shows the results of the tests in the ``bob.example.project``
package. Ideally, you should write test units for each function of your
This example shows the results of the tests in the ``bob.example.project`` package. Ideally, you should package.
write test units for each function of your package ...
.. note:: .. note::
You should put additional packages needed for testing (e.g. ``nosetests``) You should put additional packages needed for testing (e.g. ``pytest``)
in the ``test-requirements.txt`` file. in the ``test-requirements.txt`` file.
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment