baselines.rst 5.48 KB
Newer Older
André Anjos's avatar
André Anjos committed
1 2 3 4 5 6 7 8 9 10 11 12 13
.. vim: set fileencoding=utf-8 :
.. date: Thu Sep 20 11:58:57 CEST 2012

.. _bob.bio.vein.baselines:

===============================
 Executing Baseline Algorithms
===============================

The first thing you might want to do is to execute one of the vein
recognition algorithms that are implemented in ``bob.bio.vein``.


14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
Setting up Databases
--------------------

In order to run vein recognition algorithms using this package, you'll need to
make sure to download the raw files corresponding to the databases you'd like
to process. The raw files are not distributed with Bob_ software as biometric
data is, to most countries, considered sensible data that cannot be obtained
without explicit licensing from a data controller. You must visit the websites
below, sign the license agreements and then download the data before trying out
to run the baselines.

.. note::

   If you're at the Idiap Research Institute in Switzlerand, the datasets in
   the baselines mentioned in this guide are already downloaded and
   pre-installed on our shared file system. You don't need to re-download
   databases or create a ``~/.bob_bio_databases.txt`` file.


The current system readily supports the following freely available datasets:
André Anjos's avatar
André Anjos committed
34

35 36 37
* ``vera``: `Vera Fingervein`_
* ``utfvp``: `UTFVP`_
* ``put``: `PUT`_ Vein Dataset
André Anjos's avatar
André Anjos committed
38

39 40 41 42 43

After downloading the databases, annotate the base directories in which they
are installed. Then, follow the instructions in
:ref:`bob.bio.base.installation` to let this framework know where databases are
located on your system.
André Anjos's avatar
André Anjos committed
44 45 46 47 48 49 50 51 52 53 54 55 56 57


Running Baseline Experiments
----------------------------

To run the baseline experiments, you can use the ``./bin/verify.py`` script by
just going to the console and typing:

.. code-block:: sh

   $ ./bin/verify.py


This script is explained in more detail in :ref:`bob.bio.base.experiments`.
58 59
The ``./bin/verify.py --help`` option shows you, which other options you can
set.
André Anjos's avatar
André Anjos committed
60 61

Usually it is a good idea to have at least verbose level 2 (i.e., calling
62 63
``./bin/verify.py --verbose --verbose``, or the short version ``./bin/verify.py
-vv``).
André Anjos's avatar
André Anjos committed
64

65
.. note:: **Running in Parallel**
André Anjos's avatar
André Anjos committed
66

67 68 69
   To run the experiments in parallel, you can define an SGE grid or local host
   (multi-processing) configurations as explained in
   :ref:`running_in_parallel`.
André Anjos's avatar
André Anjos committed
70

71 72 73 74
   In short, to run in the Idiap SGE grid, you can simply add the ``--grid``
   command line option, without parameters. To run experiments in parallel on
   the local machine, simply add a ``--parallel <N>`` option, where ``<N>``
   specifies the number of parallel jobs you want to execute.
André Anjos's avatar
André Anjos committed
75 76


77
In the remainder of this section we introduce baseline experiments you can
78 79
readily run with this tool without further configuration. Baselines examplified
in this guide were published in [TVM14]_.
André Anjos's avatar
André Anjos committed
80 81


82 83
Repeated Line-Tracking with Miura Matching
==========================================
André Anjos's avatar
André Anjos committed
84

85 86
You can find the description of this method on the paper from Miura *et al.*
[MNM04]_.
André Anjos's avatar
André Anjos committed
87

88
To run the baseline on the `VERA fingervein`_ database, using the ``NOM``
89
protocol (called ``Full`` in [TVM14]_), do the following:
André Anjos's avatar
André Anjos committed
90

91

92
.. code-block:: sh
André Anjos's avatar
André Anjos committed
93

94 95
   $ ./bin/verify.py --database=vera --protocol=NOM --preprocessor=nopp --extractor=repeatedlinetracking --algorithm=match-rlt --sub-directory="rlt" --verbose --verbose

André Anjos's avatar
André Anjos committed
96

97 98 99
.. tip::

   If you have more processing cores on your local machine and don't want to
100 101 102
   submit your job for SGE execution, you can run it in parallel (using 4
   parallel tasks) by adding the options ``--parallel=4 --nice=10``.

103 104 105 106 107

This command line selects and runs the following implementations for the
toolchain:

* Database: Use the base Bob API for the VERA database implementation,
108
  protocol variant ``NOM`` which corresponds to the ``Full`` evaluation
109
  protocol described in [TVM14]_
110 111
* Preprocessor: Simple finger cropping, with no extra post-processing, as
  defined in [LLP09]_
112 113 114 115
* Feature extractor: Repeated line tracking, as explained in [MNM04]_
* Matching algorithm: "Miura" matching, as explained on the same paper
* Subdirectory: This is the subdirectory in which the scores and intermediate
  results of this baseline will be stored.
André Anjos's avatar
André Anjos committed
116 117


118
As the tool runs, you'll see printouts that show how it advances through
119 120 121 122 123 124
preprocessing, feature extraction and matching. To complete the evaluation,
run the commands bellow, that will output the equal error rate (EER) and plot
the detector error trade-off (DET) curve with the performance:

.. code-block:: sh

125
   $ ./bin/bob_eval_threshold.py  --scores <path-to>/vera/rlt/NOM/nonorm/scores-dev --criterium=eer
126 127 128 129
   ('Threshold:', 0.32023322499999995)
   FAR : 24.318% (46866/192720)
   FRR : 24.318% (107/440)
   HTER: 24.318%
130
   $ ./bin/evaluate.py --dev-files <path-to>/vera/rlt/NOM/nonorm/scores-dev --det det.pdf -l "vera-nom-mnm04" -rr
131 132
   The Recognition Rate of the development set of 'rlt' is 48.409%

133 134 135 136
To view the DET curve stored in the output file, do the following (on a Linux
machine):

.. code-block:: sh
137 138

   $ xdg-open det.pdf #to view the DET curve
André Anjos's avatar
André Anjos committed
139 140


141 142
Available Resources
-------------------
André Anjos's avatar
André Anjos committed
143

144 145 146 147 148
This package provides various different ``bob.bio.base`` resource
configurations to handle a variety of techniques in vein recognition: database
adaptors, preprocessors (cropping and illumination
normalization), feature extractors and matching algorithms. In order to list
each contribution, use the script ``./bin/resources.py``.
André Anjos's avatar
André Anjos committed
149

150
For available resources:
André Anjos's avatar
André Anjos committed
151

152
  .. code-block:: sh
André Anjos's avatar
André Anjos committed
153

154
     $ ./bin/resources.py --packages=bob.bio.vein
André Anjos's avatar
André Anjos committed
155

156 157
For a detailed explanation and the configurability of each resource, consult
:ref:`bob.bio.vein.api`.
André Anjos's avatar
André Anjos committed
158 159 160


.. include:: links.rst