From 5af2046ffeab9d1af8c23afc91b06d849f971545 Mon Sep 17 00:00:00 2001
From: Zohreh MOSTAANI <zohreh.mostaani@idiap.ch>
Date: Thu, 19 Sep 2019 16:06:45 +0200
Subject: [PATCH] [docs] combining templates and additional consideration in
one
---
doc/additional.rst | 123 ---------------------------------------
doc/development.rst | 16 +-----
doc/index.rst | 3 +-
doc/templates.rst | 137 +++++++++++++++++++++++++++++++++++++++++---
4 files changed, 133 insertions(+), 146 deletions(-)
delete mode 100644 doc/additional.rst
diff --git a/doc/additional.rst b/doc/additional.rst
deleted file mode 100644
index a4c7584d..00000000
--- a/doc/additional.rst
+++ /dev/null
@@ -1,123 +0,0 @@
-=========================
-Additional considerations
-=========================
-
-
-
-Unit tests
-----------
-
-Writing unit tests is an important asset on code that needs to run in different platforms and a great way to make sure all is OK.
-Test units are run with nose_.
-To run the test units on your package call:
-
-.. code-block:: sh
-
- $ ./bin/nosetests -v
- bob.example.library.test.test_reverse ... ok
-
- ----------------------------------------------------------------------
- Ran 1 test in 0.253s
-
- OK
-
-This example shows the results of the tests in the ``bob.example.project`` package. Ideally, you should
-write test units for each function of your package ...
-
-.. note::
-
- You should put additional packages needed for testing (e.g. ``nosetests``)
- in the ``test-requirements.txt`` file.
-
-
-Continuous integration (CI)
----------------------------
-
-.. note::
-
- This is valid for people at Idiap (or external bob contributors with access to Idiap's gitlab)
-
-.. note::
-
- Before going into CI, you should make sure that your pacakge has a gitlab repository.
- If not, do the following in your package root folder:
-
- .. code-block:: sh
-
- $ git init
- $ git remote add origin git@gitlab.idiap.ch:bob/`basename $(pwd)`
- $ git add bob/ buildout.cfg COPYING doc/ MANIFEST.IN README.rst requirements.txt setup.py version.txt
- $ git commit -m '[Initial commit]'
- $ git push -u origin master
-
-
-Copy the appropriate yml template for the CI builds:
-
-
-.. code-block:: sh
-
- # for pure python
- $ curl -k --silent https://gitlab.idiap.ch/bob/bob.admin/raw/master/templates/ci-for-python-only.yml > .gitlab-ci.yml
- # for c/c++ extensions
- $ curl -k --silent https://gitlab.idiap.ch/bob/bob.admin/raw/master/templates/ci-for-cxx-extensions.yml | tr -d '\r' > .gitlab-ci.yml
-
-Add the file to git:
-
-.. code-block:: sh
-
- $ git add .gitlab-ci.yml
-
-
-The ci file should work out of the box. It is long-ish, but generic to any
-package in the system.
-
-You also need to enable the following options - through gitlab - on your project:
-
-1. In the project "Settings" page, make sure builds are enabled
-2. If you have a private project, check the package settings and make sure that
- the "Deploy Keys" for our builders (all `conda-*` related servers) are
- enabled
-3. Visit the "Runners" section of your package settings and enable all conda
- runners, for linux and macosx variants
-4. Go into the "Variables" section of your package setup and **add common
- variables** corresponding to the usernames and passwords for uploading
- wheels and documentation tar balls to our (web DAV) server, as well as PyPI
- packages. You can copy required values from [the "Variables" section of
- bob.admin](https://gitlab.idiap.ch/bob/bob.admin/variables). N.B.: You
- **must** be logged into gitlab to access that page.
-5. Make sure to **disable** the service "Build e-mails" (those are very
- annoying)
-6. Setup the coverage regular expression under "CI/CD pipelines" to have the
- value `^TOTAL.*\s+(\d+\%)$`, which is adequate for figuring out the output
- of `coverage report`
-
-
-Python package namespace
-------------------------
-
-We like to make use of namespaces to define combined sets of functionality that go well together.
-Python package namespaces are `explained in details here <http://peak.telecommunity.com/DevCenter/setuptools#namespace-package>`_ together with implementation details.
-For bob packages, we usually use the ``bob`` namespace, using several sub-namespaces such as ``bob.io``, ``bob.ip``, ``bob.learn``, ``bob.db`` or (like here) ``bob.example``.
-
-
-The scripts you created should also somehow contain the namespace of the package. In our example,
-the script is named ``bob_example_project_version.py``, reflecting the namespace ``bob.example``
-
-
-
-
-Distributing your work
-----------------------
-
-To distribute a package, we recommend you use PyPI_.
-`Python Packaging User Guide <https://packaging.python.org/>`_ contains details
-and good examples on how to achieve this.
-Moreover, you can provide a conda_ package for your PyPI_ package for easier
-installation. In order to create a conda_ package, you need to create a conda_
-recipe for that package.
-
-For more detailed instructions on how to achieve this, please see the
-guidelines on `bob.template <https://gitlab.idiap.ch/bob/bob.admin/tree/master/templates>`_.
-
-
-.. include:: links.rst
diff --git a/doc/development.rst b/doc/development.rst
index 2d108f62..c7c6b86f 100644
--- a/doc/development.rst
+++ b/doc/development.rst
@@ -183,7 +183,7 @@ Before proceeding, you need to make sure that you already have a conda_ environm
Now you have an isolated conda environment named `dev` with proper channels set. For more information about conda channels refer to `conda channel documentation`_.
-The `bdt create` command assumes a directory named `conda' exists on the current directory and that it contains a file called `meta.yaml' containing the recipe for the package you want to create a development environment for.
+The `bdt create` command assumes a directory named `conda' exists on the current directory and that it contains a file called `meta.yaml` containing the recipe for the package you want to create a development environment for.
.. note::
@@ -359,19 +359,7 @@ This command will create a new bob package named "awesome-project" that includes
+-- setup.py
+-- version.txt
-
-There is a folder named `conda` that includes a file `meta.yaml`. As explained earlier this files includes the information used to prepare a proper conda environment.
-
-The folder named `bob` which should only include a file `__init__.py` at this stage is where you will put all your new code and functionality in.
-
-The folder named `doc` includes the minimum necessary information for building package documentation. The file `conf.py` is used by sphinx to build the documentation.
-
-`.gitlab-ci.yml` includes the information about building packages on the ci. We will talk about it later.
-
-
-COPYING??? MANIFEST.IN???
-
-`requirements.txt` and `setup.py` and `buildout.cfg` are used to setup the proper development environment. We will describe in more details in :ref:`bob.devtools.buildout`_. For now let's see how to use `buildout.cfg` file.
+For more information about the functionality of each file check :ref:`.. _bob.devtools.anatomy`.
In the root of your project there is a file `buildout.cfg` used by `buildout` to build your package locally. It should look like:
diff --git a/doc/index.rst b/doc/index.rst
index 162ddc77..3b4c2068 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -17,9 +17,8 @@ Documentation
install
development
- additional
- release
templates
+ release
api
ci
diff --git a/doc/templates.rst b/doc/templates.rst
index b2e88e6a..9cb41c38 100644
--- a/doc/templates.rst
+++ b/doc/templates.rst
@@ -2,9 +2,9 @@
.. _bob.devtools.templates:
-==========================
- New Package Instructions
-==========================
+===========================
+ Additional Considerations
+===========================
These instructions describe how create new packages for either Bob_ or BEAT_
and provides information on how to generate a complete, but empty package from
@@ -23,11 +23,134 @@ scratch.
changed. In case that happens, update your package by generating a new
setup and copying the relevant parts to your existing package(s).
-Create a new package
---------------------
-To create a new package, just use the command ``bdt new``. Use its ``--help``
-to get more information about options you can provide.
+Unit tests
+----------
+
+Writing unit tests is an important asset on code that needs to run in different platforms and a great way to make sure all is OK.
+Test units are run with nose_.
+To run the test units on your package call:
+
+.. code-block:: sh
+
+ $ ./bin/nosetests -v
+ bob.example.library.test.test_reverse ... ok
+
+ ----------------------------------------------------------------------
+ Ran 1 test in 0.253s
+
+ OK
+
+This example shows the results of the tests in the ``bob.example.project`` package. Ideally, you should
+write test units for each function of your package ...
+
+.. note::
+
+ You should put additional packages needed for testing (e.g. ``nosetests``)
+ in the ``test-requirements.txt`` file.
+
+
+Continuous integration (CI)
+---------------------------
+
+.. note::
+
+ This is valid for people at Idiap (or external bob contributors with access to Idiap's gitlab)
+
+.. note::
+
+ Before going into CI, you should make sure that your pacakge has a gitlab repository.
+ If not, do the following in your package root folder:
+
+ .. code-block:: sh
+
+ $ git init
+ $ git remote add origin git@gitlab.idiap.ch:bob/`basename $(pwd)`
+ $ git add bob/ buildout.cfg COPYING doc/ MANIFEST.IN README.rst requirements.txt setup.py version.txt
+ $ git commit -m '[Initial commit]'
+ $ git push -u origin master
+
+
+Copy the appropriate yml template for the CI builds:
+
+
+.. code-block:: sh
+
+ # for pure python
+ $ curl -k --silent https://gitlab.idiap.ch/bob/bob.admin/raw/master/templates/ci-for-python-only.yml > .gitlab-ci.yml
+ # for c/c++ extensions
+ $ curl -k --silent https://gitlab.idiap.ch/bob/bob.admin/raw/master/templates/ci-for-cxx-extensions.yml | tr -d '\r' > .gitlab-ci.yml
+
+Add the file to git:
+
+.. code-block:: sh
+
+ $ git add .gitlab-ci.yml
+
+
+The ci file should work out of the box. It is long-ish, but generic to any
+package in the system.
+
+You also need to enable the following options - through gitlab - on your project:
+
+1. In the project "Settings" page, make sure builds are enabled
+2. If you have a private project, check the package settings and make sure that
+ the "Deploy Keys" for our builders (all `conda-*` related servers) are
+ enabled
+3. Visit the "Runners" section of your package settings and enable all conda
+ runners, for linux and macosx variants
+4. Go into the "Variables" section of your package setup and **add common
+ variables** corresponding to the usernames and passwords for uploading
+ wheels and documentation tar balls to our (web DAV) server, as well as PyPI
+ packages. You can copy required values from [the "Variables" section of
+ bob.admin](https://gitlab.idiap.ch/bob/bob.admin/variables). N.B.: You
+ **must** be logged into gitlab to access that page.
+5. Make sure to **disable** the service "Build e-mails" (those are very
+ annoying)
+6. Setup the coverage regular expression under "CI/CD pipelines" to have the
+ value `^TOTAL.*\s+(\d+\%)$`, which is adequate for figuring out the output
+ of `coverage report`
+
+
+Python package namespace
+------------------------
+
+We like to make use of namespaces to define combined sets of functionality that go well together.
+Python package namespaces are `explained in details here <http://peak.telecommunity.com/DevCenter/setuptools#namespace-package>`_ together with implementation details.
+For bob packages, we usually use the ``bob`` namespace, using several sub-namespaces such as ``bob.io``, ``bob.ip``, ``bob.learn``, ``bob.db`` or (like here) ``bob.example``.
+
+
+The scripts you created should also somehow contain the namespace of the package. In our example,
+the script is named ``bob_example_project_version.py``, reflecting the namespace ``bob.example``
+
+
+Distributing your work
+----------------------
+
+To distribute a package, we recommend you use PyPI_.
+`Python Packaging User Guide <https://packaging.python.org/>`_ contains details
+and good examples on how to achieve this.
+Moreover, you can provide a conda_ package for your PyPI_ package for easier
+installation. In order to create a conda_ package, you need to create a conda_
+recipe for that package.
+
+For more detailed instructions on how to achieve this, please see the
+guidelines on `bob.template <https://gitlab.idiap.ch/bob/bob.admin/tree/master/templates>`_.
+
+.. .. _bob.devtools.anatomy:
+Anatomy of a new package
+------------------------
+
+There is a folder named `conda` that includes a file `meta.yaml`. As explained earlier this files includes the information used to prepare a proper conda environment.
+
+The folder named `bob` which should only include a file `__init__.py` at this stage is where you will put all your new code and functionality in.
+
+The folder named `doc` includes the minimum necessary information for building package documentation. The file `conf.py` is used by sphinx to build the documentation.
+
+`.gitlab-ci.yml` includes the information about building packages on the ci. We will talk about it later.
+
+
+COPYING??? MANIFEST.IN???
Continuous Integration and Deployment (CI)
--
GitLab