Merge branch 'vein_annotator' into 'master'

Request to merge changes made by Teo in the master branch

Some new functions introduced to the original annotator tool. New - line annotator made, made documentation.
Mostly merge is needed  because of documentation.

See merge request !1

.gitlab-ci.yml

-py27-conda-idiap:
-  script:
-    - source /idiap/group/torch5spro/conda/bin/activate bob-2.3.4-py27_0
+# This build file is defined in two parts: 1) a generic set of instructions you
+# probably **don't** need to change and 2) a part you may have to tune to your
+# project. It heavily uses template features from YAML to help you in only
+# changing a minimal part of it and avoid code duplication to a maximum while
+# still providing a nice pipeline display on your package.
+
+# 1) Generic instructions (only change if you know what you're doing)
+# -------------------------------------------------------------------
+
+# Definition of our build pipeline
+stages:
+  - build
+  - test
+  - docs
+  - wheels
+
+
+# Global variables
+variables:
+  CONDA_PREFIX: env
+
+
+# Template for the build stage
+# Needs to run on all supported architectures, platforms and python versions
+.build_template: &build_job
+  stage: build
+  before_script:
     - git clean -ffdx
-    - export TMPDIR=/var/tmp
-    - python bootstrap-buildout.py
+    - curl --silent https://gitlab.idiap.ch/bob/bob/snippets/7/raw | tr -d '\r' > bootstrap-conda.sh
+    - chmod 755 ./bootstrap-conda.sh
+    - ./bootstrap-conda.sh ${CONDA_FOLDER} ${PYTHON_VER} ${CONDA_PREFIX}
+  variables: &build_variables
+    BOB_DOCUMENTATION_SERVER: "http://www.idiap.ch/software/bob/docs/latest/bob/%s/master/"
+  script:
     - ./bin/buildout
-    - unset TMPDIR
-    - ./bin/python --version
-    - export NOSE_WITH_COVERAGE=1
-    - export NOSE_COVER_PACKAGE=bob.ip.annotator
-    - ./bin/nosetests -sv bob.ip.annotator
+    - if [ -x ./bin/bob_dbmanage.py ]; then ./bin/bob_dbmanage.py all download --force; fi
+    - ./bin/sphinx-build doc sphinx
+    - ./bin/python setup.py bdist_wheel --python-tag ${WHEEL_TAG}
+  after_script:
+    - rm -rf ${CONDA_PREFIX}
+  artifacts:
+    expire_in: 1 day
+    paths:
+      - bootstrap-conda.sh
+      - dist/
+      - sphinx/
+
+
+# Template for building on a Linux machine
+.build_linux_template: &linux_build_job
+  <<: *build_job
+  variables: &linux_build_variables
+    <<: *build_variables
+    CONDA_FOLDER: "/local/conda"
+    CFLAGS: "-D_GLIBCXX_USE_CXX11_ABI=0 -coverage"
+    CXXFLAGS: "-D_GLIBCXX_USE_CXX11_ABI=0 -coverage"
+
+
+# Template for building on a Mac OSX machine
+.build_mac_template: &macosx_build_job
+  <<: *build_job
+  variables: &macosx_build_variables
+    <<: *build_variables
+    CONDA_FOLDER: "/opt/conda"
+    MACOSX_DEPLOYMENT_TARGET: "10.9"
+    CFLAGS: "-pthread -coverage"
+    CXXFLAGS: "-pthread -coverage"
+    LDFLAGS: "-lpthread"
+
+
+# Template for the test stage - re-install from uploaded wheels
+# Needs to run on all supported architectures, platforms and python versions
+.test_template: &test_job
+  stage: test
+  before_script:
+    - ./bootstrap-conda.sh ${CONDA_FOLDER} ${PYTHON_VER} ${CONDA_PREFIX}
+    - source ${CONDA_FOLDER}/bin/activate ${CONDA_PREFIX}
+    - pip install --use-wheel --no-index --pre dist/*.whl
+  script:
+    - cd ${CONDA_PREFIX}
+    - python -c "from ${CI_PROJECT_NAME} import get_config; print(get_config())"
+    - coverage run --source=${CI_PROJECT_NAME} ./bin/nosetests -sv ${CI_PROJECT_NAME}
+    - coverage report
+    - sphinx-build -b doctest ../doc ../sphinx
+  after_script:
+    - rm -rf ${CONDA_PREFIX}
+
+
+# Template for the wheel uploading stage
+# Needs to run against one combination of python 2.x and 3.x if it is a python
+# only package, otherwise, needs to run in both pythons to all supported
+# architectures (Linux and Mac OSX 64-bit)
+.wheels_template: &wheels_job
+  stage: wheels
+  only:
+    - master
+    - tags
+  before_script:
+    - curl --silent https://gitlab.idiap.ch/bob/bob/snippets/8/raw | tr -d '\r' > upload-wheel.sh
+    - chmod 755 upload-wheel.sh
+  script:
+    - ./upload-wheel.sh
+
+
+# Template for (latest) documentation upload stage
+# Only one real job needs to do this
+.docs_template: &docs_job
+  stage: docs
+  only:
+    - master
+  before_script:
+    - curl --silent https://gitlab.idiap.ch/bob/bob/snippets/9/raw | tr -d '\r' > upload-sphinx.sh
+    - chmod 755 upload-sphinx.sh
+  script:
+    - ./upload-sphinx.sh
+
+
+# 2) Package specific instructions (you may tune this if needed)
+# --------------------------------------------------------------
+
+# Linux + Python 2.7: Builds, tests, uploads wheel
+build_linux_27:
+  <<: *linux_build_job
+  variables: &linux_27_build_variables
+    <<: *linux_build_variables
+    PYTHON_VER: "2.7"
+    WHEEL_TAG: "py27"
+  tags:
+    - conda-linux
+
+test_linux_27:
+  <<: *test_job
+  variables: *linux_27_build_variables
+  dependencies:
+    - build_linux_27
+  tags:
+    - conda-linux
+
+wheels_linux_27:
+  <<: *wheels_job
+  dependencies:
+    - build_linux_27
+  tags:
+    - conda-linux
+
+
+# Linux + Python 3.4: Builds and tests
+build_linux_34:
+  <<: *linux_build_job
+  variables: &linux_34_build_variables
+    <<: *linux_build_variables
+    PYTHON_VER: "3.4"
+    WHEEL_TAG: "py3"
+  tags:
+    - conda-linux
+
+test_linux_34:
+  <<: *test_job
+  variables: *linux_34_build_variables
+  dependencies:
+    - build_linux_34
+  tags:
+    - conda-linux
+
+
+# Linux + Python 3.5: Builds, tests, uploads wheel
+build_linux_35:
+  <<: *linux_build_job
+  variables: &linux_35_build_variables
+    <<: *linux_build_variables
+    PYTHON_VER: "3.5"
+    WHEEL_TAG: "py3"
+  tags:
+    - conda-linux
+
+test_linux_35:
+  <<: *test_job
+  variables: *linux_35_build_variables
+  dependencies:
+    - build_linux_35
+  tags:
+    - conda-linux
+
+wheels_linux_35:
+  <<: *wheels_job
+  dependencies:
+    - build_linux_35
+  tags:
+    - conda-linux
+
+docs_linux_35:
+  <<: *docs_job
+  dependencies:
+    - build_linux_35
+  tags:
+    - conda-linux
+
+
+# Mac OSX + Python 2.7: Builds and tests
+build_macosx_27:
+  <<: *macosx_build_job
+  variables: &macosx_27_build_variables
+    <<: *macosx_build_variables
+    PYTHON_VER: "2.7"
+    WHEEL_TAG: "py27"
+  tags:
+    - conda-macosx
+
+test_macosx_27:
+  <<: *test_job
+  variables: *macosx_27_build_variables
+  dependencies:
+    - build_macosx_27
+  tags:
+    - conda-macosx
+
+
+# Mac OSX + Python 3.4: Builds and tests
+build_macosx_34:
+  <<: *macosx_build_job
+  variables: &macosx_34_build_variables
+    <<: *macosx_build_variables
+    PYTHON_VER: "3.4"
+    WHEEL_TAG: "py3"
+  tags:
+    - conda-macosx
+
+test_macosx_34:
+  <<: *test_job
+  variables: *macosx_34_build_variables
+  dependencies:
+    - build_macosx_34
+  tags:
+    - conda-macosx
+
+
+# Mac OSX + Python 3.5: Builds and tests
+build_macosx_35:
+  <<: *macosx_build_job
+  variables: &macosx_35_build_variables
+    <<: *macosx_build_variables
+    PYTHON_VER: "3.5"
+    WHEEL_TAG: "py3"
+  tags:
+    - conda-macosx
+
+test_macosx_35:
+  <<: *test_job
+  variables: *macosx_35_build_variables
+  dependencies:
+    - build_macosx_35
   tags:
-    - lidiap2015
+    - conda-macosx

MANIFEST.in

-include README.rst bootstrap-buildout.py buildout.cfg requirements.txt version.txt
+include README.rst bootstrap-buildout.py buildout.cfg develop.cfg requirements.txt version.txt
 recursive-include bob/ip/annotator/data *.jpg

bob/ip/annotator/__init__.py

+#!/usr/bin/env python
+# vim: set fileencoding=utf-8 :
+
+def get_config():
+  """Returns a string containing the configuration information.
+  """
+  import bob.extension
+  return bob.extension.get_config(__name__)
+
+
+# gets sphinx autodoc done right - don't remove it
+__all__ = [_ for _ in dir() if not _.startswith('_')]

bob/ip/annotator/gui.py

@@ -61,6 +61,9 @@ from . import io as annotator_io
 import logging
 logger = logging.getLogger(__name__)
 
+# image filters:
+from skimage import exposure
+
 COLOR_ACTIVE = "red"
 COLOR_INACTIVE = "green"
 SHIFT = 0x0001
@@ -164,14 +167,17 @@ class AnnotatorApp(tkinter.Tk):
 
   """
 
-  def __init__(self, filelist, outputdir, readonly=False, zoom=1.0,
-      marker_radius=1, pixel_skip=5, *args, **kwargs):
+  def __init__(self, filelist, outputdir, readonly=False, zoom=1.0, basedir = None,
+               imfilter=False,marker_radius=1, pixel_skip=5, *args, **kwargs):
 
     tkinter.Tk.__init__(self, *args, **kwargs)
     self.title("annotate")
 
     # setup parameters
     filelist = _uniq(filelist)
+    if basedir:
+        self.basedir = basedir
+    else:
         if len(filelist) == 1:
           self.basedir = os.path.dirname(filelist[0])
         else:
@@ -193,6 +199,7 @@ class AnnotatorApp(tkinter.Tk):
     self.annotation_widgets = [] #the widgets related to the annotations
     self.mask_widget = None #the interpolated mask
     self.canvas = None #the main canvas
+    self.filter = imfilter
 
     self.update_image()
 
@@ -228,10 +235,22 @@ class AnnotatorApp(tkinter.Tk):
     bob_image = bob.io.base.load(self.filelist[self.curr_pos])
     if len(bob_image.shape) == 3:
       bob_image = bob_image.transpose(1,2,0)
+    #added a simple image histogram streching:
+    bob_image = numpy.float32(bob_image)
+    bob_image = (bob_image - bob_image.min()) / (bob_image.max() - bob_image.min())
+    bob_image = numpy.uint8(bob_image * 255)
     self.image = Image.fromarray(bob_image)
     if self.zoom != 1.0:
       shape = self.zoom_coordinates((self.image.width, self.image.height))
       self.image = self.image.resize(shape, Image.ANTIALIAS)
+      
+      
+      
+    if self.filter == True:
+        self.image = numpy.array(self.image)
+        self.image = exposure.equalize_adapthist(self.image, clip_limit=0.01)
+        self.image = self.image * 255
+        self.image = Image.fromarray(self.image)
     self.curr_photo = ImageTk.PhotoImage(self.image, Image.ANTIALIAS)
 
     if self.canvas is None:

bob/ip/annotator/io.py

@@ -10,6 +10,78 @@ import six
 import numpy
 
 
+def save_multiple(data, fp, backup=False):
+  """Saves a given data set to a file
+
+
+  Parameters:
+
+  data : Data array(s) to be saved;
+    In case of line annotations it is a tuple of lists of the keypoints (x, y).
+
+  fp (File, str): File or filename to which the data is saved.
+    The extension should be ``*.npy``
+
+  backup (boolean, Optional): If set, backs-up a possibly existing file path
+    before overriding it. Note this is not valid in case 'fp' above points to
+    an opened file.
+    
+    
+    
+    Example with ``numpy.sevez`` that is used in this function:
+        data = ([(1, 2), (2, 3)], [(5,6)])
+        fp = open("test.npz", 'wt')
+        numpy.savez(fp, data)
+        fp.close()
+        #read data:
+        fp = open("test.npz", 'rt')
+        npzdata = numpy.load(fp)
+        npzdata.files
+        npzdata['arr_0']
+  """
+
+  if isinstance(fp, six.string_types):
+
+    if backup and os.path.exists(fp):
+      bname = fp + '~'
+      if os.path.exists(bname): os.unlink(bname)
+      os.rename(fp, bname)
+
+    fp = open(fp, 'wt')
+
+    numpy.savez(fp, data)#numpy.savetxt(fp, data, fmt='%d')
+    
+    fp.close()
+  return 0
+
+def load_multiple(fp):
+  """Loads a given data set from ``*.npy`` file
+
+  Parameters:
+
+    fp (File, str): The name of a file, with full path, to be used for
+      reading the data or an already opened file-like object, that accepts
+      the "read()" call.
+
+  Returns:
+
+    tuple: Containing the lists of line points (annotations), in the
+      format (y, x). That is Bob's style.
+
+  """
+
+  if isinstance(fp, six.string_types):
+    fp = open(fp, 'rt')
+  retval = numpy.load(fp)
+  retval = retval['arr_0'].tolist()
+  
+  fp.close()
+  #retval = numpy.loadtxt(fp, ndmin=2)
+  return retval#list([tuple(k) for k in retval])
+
+
+
+
 def save(data, fp, backup=False):
   """Saves a given data set to a file
 

bob/ip/annotator/script/annotate.py

@@ -4,7 +4,7 @@
 """A TkInter-based keypoint annotation tool for images (%(version)s)
 
 Usage:
-  %(prog)s [-v ...] [-r] [-z <value] [-o=<path>] <image>...
+  %(prog)s [-v ...] [-r] [-f] [-s] [-z <value>] [-b=<path>] [-o=<path>] <image>...
 
 
 Arguments:
@@ -32,6 +32,12 @@ Options:
                        Values larger than 1.0 will make the image bigger than
                        what it really is, while values between 0 and 1.0 will
                        make it smaller. [default: 1.0]
+  -f, --imfilter       If set, images are filtered using adaptive hystogram.
+  -s, --subdir         If set, all files in the subdirectories with extension
+                       set after the "*" will be included.
+  -b, --basedir=<path> If given, use this as the basedir instead of basedir 
+                       acquired from image paths.
+
 
 
 Examples:
@@ -93,7 +99,19 @@ def main(user_input=None):
   args['--output'] = os.path.abspath(os.path.expanduser(args['--output']))
   args['<image>'] = [os.path.abspath(k) for k in args['<image>']]
 
+  if args['--subdir'] == True:
+      temp = []
+      for paths in args['<image>']:
+          splitted_path =  paths.split("*")
+          if len(splitted_path) != 2:
+              raise IOError("When -s flag is set, file path and it's extension, seperated with '*' is expected, e.g.: 'a/b/*.png'")
+          for path, subdirs, files in os.walk(splitted_path[0]):
+              for name in files:
+                  if name.endswith(splitted_path[1]):
+                      temp.append(os.path.join(path, name))
+      args['<image>'] = temp
+  
   app = AnnotatorApp(args['<image>'], args['--output'], args['--readonly'],
-      float(args['--zoom']))
+      float(args['--zoom']),args['--basedir'], args['--imfilter'])
   app.mainloop()
   return 0

bob/ip/annotator/script/annotate_line.py

+#!/usr/bin/env python
+# vim: set fileencoding=utf-8 :
+
+"""A TkInter-based keypoint annotation tool for images (%(version)s)
+
+Usage:
+  %(prog)s [-v ...] [-r] [-s] [-z <value>] [-a=<path>] [-b=<path>] [-o=<path>] [-c] <image>...
+
+
+Arguments:
+  <image>  Paths to the images to be annotated. The paths given will be made
+           absolute and duplicates will be removed. The common directory
+           between all images is saved as a base directory for the application.
+           The output directory will contain the annotations taking into
+           consideration the path structure of input images starting from that
+           base directory. For example, if input images are "A/B/img1.png" and
+           "A/C/img2.png", then the output annotations will be available in
+           "<output-path>/B/img1.txt" and "<output-path>/C/img2.txt"
+
+
+Options:
+  -h, --help           Shows this help message and exit
+  -V, --version        Shows program's version number and exit
+  -o, --output=<path>  The output directory where annotations will be saved
+                       [default: .]
+  -a,--annotdir=<path> The base directory where annotations are saved for given
+                       input files. It is assumed that annotation folder-structure
+                       are the same as for the images.
+  -v, --verbose        If set, increases the verbosity level - may be given
+                       twice, to enable debug output
+  -r, --readonly       If set, then only visualizes current annotations (does
+                       not overwrite existing ones)
+  -c --cutroi          If set, ROI is cut from the input image.
+  -z, --zoom=<value>   Provides the zoom multiplier for the input image, as a
+                       floating number. This number has to be greater than 0.0.
+                       Values larger than 1.0 will make the image bigger than
+                       what it really is, while values between 0 and 1.0 will
+                       make it smaller. [default: 1.0]
+  -s, --subdir         If set, all files in the subdirectories with extension
+                       set after the "*" will be included.
+  -b, --basedir=<path> If given, use this as the basedir instead of basedir 
+                       acquired from image paths.
+
+
+Examples:
+
+  Annotate two images, results in the directory `./tmp':
+
+    $ %(prog)s -v --output=./tmp A/B/img1.png A/C/img2.png
+
+  Annotate all images, results in the directory `./tmp':
+
+    $ %(prog)s -v --output=./tmp A/*/*.png
+
+  Annotate the image of Lena, with a 2x zoom, get the result in `lena.txt`:
+
+    $ %(prog)s -v --zoom=2 bob/ip/annotator/data/lena.jpg
+
+  Visualize annotations for Lena, with a 0.5x zoom, from the file `lena.txt`:
+
+    $ %(prog)s -v --readonly --zoom=0.5 bob/ip/annotator/data/lena.jpg
+
+"""
+
+import os
+import sys
+import pkg_resources
+
+import logging
+logger = logging.getLogger(__name__)
+
+from docopt import docopt
+
+from ..line_gui import LineAnnotatorApp
+
+
+def main(user_input=None):
+
+  # Parse the command-line arguments
+  if user_input is not None:
+    arguments = user_input
+  else:
+    arguments = sys.argv[1:]
+
+  completions = dict(
+      prog = os.path.basename(sys.argv[0]),
+      version = pkg_resources.require('bob.ip.annotator')[0].version,
+      )
+
+  args = docopt(
+      __doc__ % completions,
+      argv=arguments,
+      options_first=True,
+      version=completions['version'],
+      )
+
+  # If the user wants more verbosity, lower the level
+  if args['--verbose'] == 1: logging.getLogger().setLevel(logging.INFO)
+  elif args['--verbose'] >= 2: logging.getLogger().setLevel(logging.DEBUG)
+
+  args['--output'] = os.path.abspath(os.path.expanduser(args['--output']))
+  args['<image>'] = [os.path.abspath(k) for k in args['<image>']]
+  
+  if args['--subdir'] == True:
+      temp = []
+      for paths in args['<image>']:
+          splitted_path =  paths.split("*")
+          if len(splitted_path) != 2:
+              raise IOError("When -s flag is set, file path and it's extension, seperated with '*' is expected, e.g.: 'a/b/*.png'")
+          for path, subdirs, files in os.walk(splitted_path[0]):
+              for name in files:
+                  if name.endswith(splitted_path[1]):
+                      temp.append(os.path.join(path, name))
+      args['<image>'] = temp
+
+  app = LineAnnotatorApp(args['<image>'], args['--annotdir'], args['--output'], args['--readonly'],
+      float(args['--zoom']), args['--cutroi'], args['--basedir'])
+  app.mainloop()
+  return 0

bootstrap-buildout.py

@@ -49,8 +49,8 @@ parser.add_option("--version",
 parser.add_option("-t", "--accept-buildout-test-releases",
                   dest='accept_buildout_test_releases',
                   action="store_true", default=False,
-                  help=("Normally, if you do not specify a --buildout-version, "
-                        "the bootstrap script and buildout gets the newest "
+                  help=("Normally, if you do not specify a --version, the "
+                        "bootstrap script and buildout gets the newest "
                         "*final* versions of zc.buildout and its recipes and "
                         "extensions for you.  If you use this flag, "
                         "bootstrap and buildout will get the newest releases "

buildout.cfg

+; vim: set fileencoding=utf-8 :
+; Fri 19 Aug 16:45:02 CEST 2016
+
 [buildout]
 parts = scripts
-extensions = bob.buildout
-eggs = bob.ip.annotator
 develop = .
+eggs = bob.ip.annotator
+extensions = bob.buildout
 newest = false
+verbose = true
 
 [scripts]
 recipe = bob.buildout:scripts
+dependent-scripts = true
\ No newline at end of file

develop.cfg

+[buildout]
+parts = scripts
+
+eggs = bob.ip.annotator
+       bob.db.biowave_test
+       bob.db.putvein
+       bob.db.vera
+       bob.bio.vein
+
+
+extensions = bob.buildout
+             mr.developer
+
+auto-checkout = *
+newest = false
+
+develop = src/bob.db.biowave_test
+          src/bob.db.putvein
+          src/bob.db.vera
+          src/bob.bio.vein
+          .
+
+
+[sources]
+bob.db.biowave_test = git git@gitlab.idiap.ch:bob/bob.db.biowave_test.git
+bob.db.putvein = git git@gitlab.idiap.ch:bob/bob.db.putvein.git
+bob.db.vera = git git@gitlab.idiap.ch:bob/bob.db.vera.git
+bob.bio.vein = git git@gitlab.idiap.ch:bob/bob.bio.vein.git branch=package-update
+
+
+
+[scripts]
+recipe = bob.buildout:scripts

doc/annotator-guide.rst

+.. vim: set fileencoding=utf-8 :
+.. @author: Teodors Eglitis <Teodors.Eglitis@idiap.ch>
+.. @date:   Wed Aug 17 14:47:18 CEST 2016
+
+===========================
+GUIDE: Wrist ROI annotation
+===========================
+
+In this guide is summarized all that one should need to know to be able to annotate ROI in Idiap's project BIOWAVE wrist vein images.
+
+To do that you will be using tool *annotate* from the ``non.ip.annotator`` package.
+
+All work can be divided in to 3 stages:
+1) Selecting the image *batches*;
+2) Annotating the images;
+3) Finalizing the annotation.
+
+
+Selection of image *batches*
+-----------------------------
+
+Go to the Google docs spreadsheet_ . There is listed *111 batches* with wrist vein images. Each batch contains 6 images (it should take you 3-4 minutes to complete one batch's ROI annotations). Choose batches whose images you will annotate by writing your name in the corresponding cell.
+
+Image annotation
+----------------
+
+First, open the ``terminal`` in your Idiap machine and switch ``CONDA`` environment by typing::
+    
+    source /idiap/group/torch5spro/conda/bin/activate bob-2.3.4-py27_0
+
+Now you can annotate one batch (6 images) or multiple batches. To annotate one, type::
+    
+    /idiap/project/biowave/biowave/bob.ip.annotator/bin/annotate.py -v -f -s --zoom=2 --output=/idiap/project/biowave/biowave/ROI_annotations/  --basedir=/idiap/project/biowave/biowave/annotation_images/ /idiap/project/biowave/biowave/annotation_images/BATCH1/*.png
+
+Where ``BATCH1``, your chosen batch name, e.g. ``001/Right``
+
+You can also run several batches, by replacing::
+    
+    BATCH1
+
+with::
+
+    {BATCH1,BATCH2,BATCHn}
+
+**Be careful -- no spaces between batch names!** E.g., if I want to annotate batches ``001/Right``, ``001/Left``, ``002/Right``, the complete command is::
+    
+        /idiap/project/biowave/biowave/bob.ip.annotator/bin/annotate.py -v -f -s --zoom=2 --output=/idiap/project/biowave/biowave/ROI_annotations/  --basedir=/idiap/project/biowave/biowave/annotation_images/ /idiap/project/biowave/biowave/annotation_images/{001/Right,001/Left,002/Right}/*.png
+
+**Again remember -- no spaces!**
+
+After running the command an GUI will open. Now you can start to mark ROI in the images:
+
+1) Start from the top left of the image;
+2) Click to add as many points as you find sufficient to match the wrist border - as you draw the points, you'll see a polygon being formed indicating the area you're selecting. It is better if the border appears outside of the polygon (ROI);
+3) Make sure you get the whole wrist ROI, as accurately as possible (30 seconds to 1 minute per image) should be enough.
+4) If you make a mistake, press the ``d`` key to remove last point;
+5) Use the arrow keys on your keyboard to fine-tune the point location, in case your mouse click was not precise enough (only works for the last annotated point, in red)
+6) Use the mouse scrollwheel (down), to move to the next image in the batch.
+7) Once you're done with the batch (scrolling down does not change the image anymore), hit ``q`` to quit the program
+8) All the above is explained in the help dialogue of the annotator, hit ``?`` to get this window.
+
+
+When you scroll the scrollwheel as well as when you exit the GUI by pressing ``q``, your annotations are saved. You can go back (by scrolling the scrollwheel or opening the GUI again) and continue to add annotations (points), delete one or more of them (by pressing ``d`` multiple times) or delete the image's annotation completely by pressing ``D`` key.
+
+The number of points in each annotation is arbitrary - we just need to the get shape of the illuminated wrist area right.
+
+Example:
+
+.. image:: img/ROI.png
+
+
+To see more examples, in ``terminal`` run command (this command opens annotations for first 2 batches in the read-only mode)::
+    
+    /idiap/project/biowave/biowave/bob.ip.annotator/bin/annotate.py -v -f -s --readonly --zoom=2 --output=/idiap/project/biowave/biowave/ROI_annotations/  --basedir=/idiap/project/biowave/biowave/annotation_images/ /idiap/project/biowave/biowave/annotation_images/{001/Right,001/Left}/*.png
+
+
+Finalization of annotation
+--------------------------
+
+When you are finished all annotations:
+
+1) Go back to the Google spreadsheet_ and mark that you are done with the ROI's annotation;
+2) Run this command in the console (this command gives write access to all people in the group, so I can move/edit the files if needed)::
+    
+    find /idiap/project/biowave/biowave/ROI_annotations/ -user $USER -exec chmod g+rw {} \;
+
+
+That's it!
+
+.. _spreadsheet: https://docs.google.com/spreadsheets/d/1-YcOitDkGDL4T0eccdkAQ0RdfqplzPvVrX-WcL8dUS8/edit?usp=sharing
+

doc/annotator.rst

+.. vim: set fileencoding=utf-8 :
+.. @author: Teodors Eglitis <Teodors.Eglitis@idiap.ch>
+.. @date:   Wed Aug 17 14:47:18 CEST 2016
+
+===============
+Annotation Tool
+===============
+
+Docs for keypoint annotation tool
+
+
+.. automodule:: bob.ip.annotator

doc/conf.py

+#!/usr/bin/env python
+# vim: set fileencoding=utf-8 :
+
+import os
+import sys
+import glob
+import pkg_resources
+
+
+# -- General configuration -----------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+needs_sphinx = '1.3'
+
+# Add any Sphinx extension module names here, as strings. They can be extensions
+# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
+extensions = [
+    'sphinx.ext.todo',
+    'sphinx.ext.coverage',
+    'sphinx.ext.ifconfig',
+    'sphinx.ext.autodoc',
+    'sphinx.ext.autosummary',
+    'sphinx.ext.doctest',
+    'sphinx.ext.graphviz',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.napoleon',
+    'sphinx.ext.viewcode',
+    ]
+
+import sphinx
+if sphinx.__version__ >= "1.4.1":
+    extensions.append('sphinx.ext.imgmath')
+else:
+    extensions.append('sphinx.ext.pngmath')
+
+# Always includes todos
+todo_include_todos = True
+
+# Generates auto-summary automatically
+autosummary_generate = True
+
+# Create numbers on figures with captions
+numfig = True
+
+# If we are on OSX, the 'dvipng' path maybe different
+dvipng_osx = '/opt/local/libexec/texlive/binaries/dvipng'
+if os.path.exists(dvipng_osx): pngmath_dvipng = dvipng_osx
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix of source filenames.
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'bob.ip.annotator'
+import time
+copyright = u'%s, Idiap Research Institute' % time.strftime('%Y')
+
+# Grab the setup entry
+distribution = pkg_resources.require(project)[0]
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = distribution.version
+# The full version, including alpha/beta/rc tags.
+release = distribution.version
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['links.rst']
+
+# The reST default role (used for this markup: `text`) to use for all documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# Some variables which are useful for generated material
+project_variable = project.replace('.', '_')
+short_description = u'Image Key Point Annotation Tool'
+owner = [u'Idiap Research Institute']
+
+
+# -- Options for HTML output ---------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+import sphinx_rtd_theme
+html_theme = 'sphinx_rtd_theme'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = project_variable
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+html_logo = 'img/logo.png'
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+html_favicon = 'img/favicon.ico'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+#html_static_path = ['_static']
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = project_variable + u'_doc'
+
+
+# -- Post configuration --------------------------------------------------------
+
+# Included after all input documents
+rst_epilog = """
+.. |project| replace:: Bob
+.. |version| replace:: %s
+.. |current-year| date:: %%Y
+""" % (version,)
+
+# Default processing flags for sphinx
+autoclass_content = 'class'
+autodoc_member_order = 'bysource'
+autodoc_default_flags = [
+  'members',
+  'undoc-members',
+  'inherited-members',
+  'show-inheritance',
+  ]
+
+# For inter-documentation mapping:
+from bob.extension.utils import link_documentation
+intersphinx_mapping = link_documentation()
+
+# We want to remove all private (i.e. _. or __.__) members
+# that are not in the list of accepted functions
+accepted_private_functions = ['__array__']
+
+def member_function_test(app, what, name, obj, skip, options):
+  # test if we have a private function
+  if len(name) > 1 and name[0] == '_':
+    # test if this private function should be allowed
+    if name not in accepted_private_functions:
+      # omit privat functions that are not in the list of accepted private functions
+      return skip
+    else:
+      # test if the method is documented
+      if not hasattr(obj, '__doc__') or not obj.__doc__:
+        return skip
+  return False
+
+def setup(app):
+  app.connect('autodoc-skip-member', member_function_test)

doc/index.rst

+.. vim: set fileencoding=utf-8 :
+.. @author: Teodors Eglitis <Teodors.Eglitis@idiap.ch>
+.. @date:   Wed Aug 17 14:47:18 CEST 2016
+
+.. _bob.ip.annotator:
+
+=======================================
+Image Key Point Annotation Tool
+=======================================
+
+.. todolist::
+
+Documentation
+-------------
+
+.. toctree::
+   :maxdepth: 2
+
+   annotator
+   annotator-guide
+   line_annotator
+   line-annotator-guide
+
+Indices and tables
+------------------
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
+
+.. automodule:: bob.ip.annotator
+

doc/line-annotator-guide.rst

+.. vim: set fileencoding=utf-8 :
+.. @author: Teodors Eglitis <Teodors.Eglitis@idiap.ch>
+.. @date:   Wed Aug 17 14:47:18 CEST 2016
+
+============================
+GUIDE: Wrist VEIN annotation
+============================
+
+In this guide is summarized all that one should need to know to be able to annotate ROI in Idiap's project BIOWAVE wrist vein images.
+
+To do that you will be using tool ``annotate_line.py`` from the ``non.ip.annotator`` package.
+
+All work can be divided in to 3 stages:
+1) Selecting the image *batches*;
+2) Annotating the images;
+3) Finalizing the annotation.
+
+
+Selection of image *batches*
+-----------------------------
+
+Go to the Google docs spreadsheet_ . There is listed *111 batches* with wrist vein images. Each batch contains 6 images (it should take you 10-12 minutes to complete one batch's vein annotations). Choose batches whose images you will annotate by writing your name in the corresponding cell. **Careful -- to be able to annotate lines first ROI annotation must be done -- choose only batches which ROI annotation is finished.**
+If you get an error ``IOError: No annotation for file...`` it can mean that:
+
+1) No annotations is done for the current path;
+2) the one responsible for the ROI annotation hasn't executed the ``chmod`` command (please see the GUIDE about ROI annotation)
+
+Image annotation
+----------------
+
+First, open the ``terminal`` in your Idiap machine and switch ``CONDA`` environment by typing::
+    
+    source /idiap/group/torch5spro/conda/bin/activate bob-2.3.4-py27_0
+
+Now you can annotate one batch (6 images) or multiple batches. To annotate one, type::
+    
+        /idiap/project/biowave/biowave/bob.ip.annotator/bin/annotate_line.py -c -s -vv --zoom=2 --annotdir=/idiap/project/biowave/biowave/ROI_annotations/ --output=/idiap/project/biowave/biowave/VEIN_annotations --basedir=/idiap/project/biowave/biowave/annotation_images/ /idiap/project/biowave/biowave/annotation_images/BATCH1/*.png
+
+Where ``BATCH1``, your chosen batch name, e.g. ``001/Right``
+
+You can also run several batches, by replacing::
+    
+    BATCH1
+
+with::
+
+    {BATCH1,BATCH2,BATCHn}
+
+**Be careful -- no spaces between batch names!** E.g., if I want to annotate batches ``001/Right``, ``001/Left``, ``002/Right``, the complete command is::
+    
+        /idiap/project/biowave/biowave/bob.ip.annotator/bin/annotate_line.py -c -s -vv --zoom=2 --annotdir=/idiap/project/biowave/biowave/ROI_annotations/ --output=/idiap/project/biowave/biowave/VEIN_annotations --basedir=/idiap/project/biowave/biowave/annotation_images/ /idiap/project/biowave/biowave/annotation_images/{001/Right,001/Left,002/Right}/*.png
+
+**Again remember -- no spaces!**
+
+After running the command an GUI will open. Now you can start to mark Veins in the images.
+
+In this GUI image filtering is introduced. Now, when GUI is opened and an image is displayed, you can circle through image filters by pressing key ``f``. You are able to see unfiltered image and image filtered using Contrast Limited Adaptive Histogram Equalization (CLAHE) (more information_) with 3 different parameters. With each press of key ``f`` (stands for *filtering*) GUI circles through filters -- *unfiltered image* -> *filter 1* -> *filter 2* -> *filter 3* -> *unfiltered image*.
+
+**Attention** -- although *CLAHE* can highlight the vein pattern, this non-linear operation can also introduce artefacts to the images and it *may* seem that there is some vein pattern in the image, when in reality it is an artefact caused by noise or a smooth gradient in the input image. That is why I recommend that you circle through all the filtered images and the original one before you mark some region as *vein*.
+
+
+This GUI is similar with the previous one, but now the task is to mark the lines. That is why now you are able to annotate multiple lines in one image.
+
+For the first line (vein) - start clicking using **left** mouse button. The *active* line is in blue colour, the active (last) point -- in red. As before, you can move the active point, using arrow keys on your keyboard. You can also delete the active point by pressing ``d`` or the whole active (blue) line by pressing ``D``.
+
+When you are finished with one vein (line), you can start to annotate next one by pressing the **right** keyboard button. When you press the right button, no annotation is set but you change the active line (you can notice that the previous-active line changes it's colour to green), so it doesn't matter where you press the right key. After you can start to annotate the second line - when pressing the **left** mouse button a new blue line (active line) will appear etc.
+
+It is also possible to *circle* through the lines by pressing the key ``c`` (stands for "circle"). By pressing the key active line is switched  -- now a different line is in blue colour, if you have annotated multiple ones and you can add points to it, delete points or delete the entire line by pressing ``D``.
+
+As before, use the mouse scrollwheel (down), to move to the next image in the batch or *up* to the previous image. Once you're done with the batch (scrolling down does not change the image anymore), hit ``q`` to quit the program. All the above is explained in the help dialogue of the annotator, hit ``?`` to get this window.
+
+The number of points per vein is arbitrary - we need to get shape of the complete vein pattern.
+
+Example:
+
+.. image:: img/VEINS.png
+
+
+To see more examples, in ``terminal`` run command (this command opens annotations for first 3 batches in the read-only mode)::
+    
+    /idiap/project/biowave/biowave/bob.ip.annotator/bin/annotate_line.py -c -s -v --readonly --zoom=2 --annotdir=/idiap/project/biowave/biowave/ROI_annotations/ --output=/idiap/project/biowave/biowave/VEIN_annotations --basedir=/idiap/project/biowave/biowave/annotation_images/ /idiap/project/biowave/biowave/annotation_images/{001/Right,001/Left,002/Right}/*.png
+
+
+Finalization of annotation
+--------------------------
+
+When you are finished all annotations:
+
+1) Go back to the Google spreadsheet_ and mark that you are done with the VEINS annotations;
+2) Run this command in the console (this command gives write access to all people in the group, so I can move/edit the files if needed)::
+    
+    find /idiap/project/biowave/biowave/VEIN_annotations/ -user $USER -exec chmod g+rw {} \;
+
+
+That's it!
+
+.. _spreadsheet: https://docs.google.com/spreadsheets/d/1-YcOitDkGDL4T0eccdkAQ0RdfqplzPvVrX-WcL8dUS8/edit?usp=sharing
+
+.. _information: https://en.wikipedia.org/wiki/Adaptive_histogram_equalization#CLAHE