Commit 97f8d8e3 authored by Andre Anjos's avatar Andre Anjos

Documentation fixes

parent ef334efd
......@@ -99,7 +99,7 @@ release = distribution.version
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['links.rst']
exclude_patterns = ['**/links.rst']
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
......@@ -243,8 +243,10 @@ latex_logo = ''
# Included after all input documents
rst_epilog = """
.. |version| replace:: %s
.. |project| replace:: Bob
.. |url| replace:: https://www.idiap.ch/software/bob/
.. |version| replace:: %s
.. |current-year| date:: %%Y
""" % (version,)
# -- Options for manual page output --------------------------------------------
......
......@@ -2,10 +2,15 @@
.. Andre Anjos <andre.dos.anjos@gmail.com>
.. Tue 15 Oct 17:41:52 2013
.. testsetup:: measuretest
.. testsetup:: *
import numpy
import xbob.measure
import numpy
positives = numpy.random.normal(1,1,100)
negatives = numpy.random.normal(-1,1,100)
import matplotlib
if not hasattr(matplotlib, 'backends'):
matplotlib.use('pdf') #non-interactive avoids exception on display
import xbob.measure
============
User Guide
......@@ -100,16 +105,6 @@ defined in the first equation.
loaded your scores in two 1D float64 vectors and are ready to evaluate the
performance of the classifier.
.. testsetup:: *
import numpy
positives = numpy.random.normal(1,1,100)
negatives = numpy.random.normal(-1,1,100)
import bob
import matplotlib
if not hasattr(matplotlib, 'backends'):
matplotlib.use('pdf') #non-interactive avoids exception on display
Evaluation
----------
......@@ -120,16 +115,16 @@ the following techniques:
>>> # negatives, positives = parse_my_scores(...) # write parser if not provided!
>>> T = 0.0 #Threshold: later we explain how one can calculate these
>>> correct_negatives = bob.measure.correctly_classified_negatives(negatives, T)
>>> correct_negatives = xbob.measure.correctly_classified_negatives(negatives, T)
>>> FAR = 1 - (float(correct_negatives.sum())/negatives.size)
>>> correct_positives = bob.measure.correctly_classified_positives(positives, T)
>>> correct_positives = xbob.measure.correctly_classified_positives(positives, T)
>>> FRR = 1 - (float(correct_positives.sum())/positives.size)
We do provide a method to calculate the FAR and FRR in a single shot:
.. doctest::
>>> FAR, FRR = bob.measure.farfrr(negatives, positives, T)
>>> FAR, FRR = xbob.measure.farfrr(negatives, positives, T)
The threshold ``T`` is normally calculated by looking at the distribution of
negatives and positives in a development (or validation) set, selecting a
......@@ -142,13 +137,13 @@ calculation of the threshold:
.. doctest::
>>> T = bob.measure.eer_threshold(negatives, positives)
>>> T = xbob.measure.eer_threshold(negatives, positives)
* Threshold for the minimum HTER
.. doctest::
>>> T = bob.measure.min_hter_threshold(negatives, positives)
>>> T = xbob.measure.min_hter_threshold(negatives, positives)
* Threshold for the minimum weighted error rate (MWER) given a certain cost
:math:`\beta`.
......@@ -156,12 +151,12 @@ calculation of the threshold:
.. code-block:: python
>>> cost = 0.3 #or "beta"
>>> T = bob.measure.min_weighted_error_rate_threshold(negatives, positives, cost)
>>> T = xbob.measure.min_weighted_error_rate_threshold(negatives, positives, cost)
.. note::
By setting cost to 0.5 is equivalent to use
:py:meth:`bob.measure.min_hter_threshold`.
:py:meth:`xbob.measure.min_hter_threshold`.
Plotting
--------
......@@ -182,7 +177,7 @@ town. To plot an ROC curve, in possession of your **negatives** and
>>> from matplotlib import pyplot
>>> # we assume you have your negatives and positives already split
>>> npoints = 100
>>> bob.measure.plot.roc(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test') # doctest: +SKIP
>>> xbob.measure.plot.roc(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test') # doctest: +SKIP
>>> pyplot.xlabel('FRR (%)') # doctest: +SKIP
>>> pyplot.ylabel('FAR (%)') # doctest: +SKIP
>>> pyplot.grid(True)
......@@ -194,7 +189,7 @@ You should see an image like the following one:
:include-source: False
As can be observed, plotting methods live in the namespace
:py:mod:`bob.measure.plot`. They work like `Matplotlib`_'s `plot()`_ method
:py:mod:`xbob.measure.plot`. They work like `Matplotlib`_'s `plot()`_ method
itself, except that instead of receiving the x and y point coordinates as
parameters, they receive the two :py:class:`numpy.ndarray` arrays with
negatives and positives, as well as an indication of the number of points the
......@@ -217,8 +212,8 @@ A DET curve can be drawn using similar commands such as the ones for the ROC cur
>>> from matplotlib import pyplot
>>> # we assume you have your negatives and positives already split
>>> npoints = 100
>>> bob.measure.plot.det(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test') # doctest: +SKIP
>>> bob.measure.plot.det_axis([0.01, 40, 0.01, 40]) # doctest: +SKIP
>>> xbob.measure.plot.det(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test') # doctest: +SKIP
>>> xbob.measure.plot.det_axis([0.01, 40, 0.01, 40]) # doctest: +SKIP
>>> pyplot.xlabel('FAR (%)') # doctest: +SKIP
>>> pyplot.ylabel('FRR (%)') # doctest: +SKIP
>>> pyplot.grid(True)
......@@ -234,20 +229,20 @@ This will produce an image like the following one:
If you wish to reset axis zooming, you must use the Gaussian scale rather
than the visual marks showed at the plot, which are just there for
displaying purposes. The real axis scale is based on the
``bob.measure.ppndf()`` method. For example, if you wish to set the x and y
``xbob.measure.ppndf()`` method. For example, if you wish to set the x and y
axis to display data between 1% and 40% here is the recipe:
.. doctest::
>>> #AFTER you plot the DET curve, just set the axis in this way:
>>> pyplot.axis([bob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)]) # doctest: +SKIP
>>> pyplot.axis([xbob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)]) # doctest: +SKIP
We provide a convenient way for you to do the above in this module. So,
optionally, you may use the ``bob.measure.plot.det_axis`` method like this:
optionally, you may use the ``xbob.measure.plot.det_axis`` method like this:
.. doctest::
>>> bob.measure.plot.det_axis([1, 40, 1, 40]) # doctest: +SKIP
>>> xbob.measure.plot.det_axis([1, 40, 1, 40]) # doctest: +SKIP
EPC
===
......@@ -257,7 +252,7 @@ the test (or evaluation) set ones. Because of this the API is slightly modified:
.. doctest::
>>> bob.measure.plot.epc(dev_neg, dev_pos, test_neg, test_pos, npoints, color=(0,0,0), linestyle='-') # doctest: +SKIP
>>> xbob.measure.plot.epc(dev_neg, dev_pos, test_neg, test_pos, npoints, color=(0,0,0), linestyle='-') # doctest: +SKIP
>>> pyplot.show() # doctest: +SKIP
This will produce an image like the following one:
......@@ -268,11 +263,11 @@ This will produce an image like the following one:
Fine-tunning
============
The methods inside :py:mod:`bob.measure.plot` are only provided as a
`Matplotlib`_ wrapper to equivalent methods in :py:mod:`bob.measure` that can
The methods inside :py:mod:`xbob.measure.plot` are only provided as a
`Matplotlib`_ wrapper to equivalent methods in :py:mod:`xbob.measure` that can
only calculate the points without doing any plotting. You may prefer to tweak
the plotting or even use a different plotting system such as gnuplot. Have a
look at the implementations at :py:mod:`bob.measure.plot` to understand how
look at the implementations at :py:mod:`xbob.measure.plot` to understand how
to use the |project| methods to compute the curves and interlace that in the
way that best suits you.
......@@ -282,8 +277,8 @@ Full applications
We do provide a few scripts that can be used to quickly evaluate a set of
scores. We present these scripts in this section. The scripts take as input
either a 4-column or 5-column data format as specified in the documentation of
:py:mod:`bob.measure.load.four_column` or
:py:mod:`bob.measure.load.five_column`.
:py:mod:`xbob.measure.load.four_column` or
:py:mod:`xbob.measure.load.five_column`.
To calculate the threshold using a certain criterion (EER, min.HTER or weighted
Error Rate) on a set, after setting up |project|, just do:
......
......@@ -4,16 +4,16 @@
"""Tutorial for plotting a DET curve
"""
import bob
import numpy
import xbob.measure
from matplotlib import pyplot
positives = numpy.random.normal(1,1,100)
negatives = numpy.random.normal(-1,1,100)
npoints = 100
bob.measure.plot.det(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test')
bob.measure.plot.det_axis([0.1, 80, 0.1, 80])
xbob.measure.plot.det(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test')
xbob.measure.plot.det_axis([0.1, 80, 0.1, 80])
pyplot.grid(True)
pyplot.xlabel('FAR (%)')
pyplot.ylabel('FRR (%)')
......
......@@ -4,8 +4,8 @@
"""Tutorial for plotting an EPC curve
"""
import bob
import numpy
import xbob.measure
from matplotlib import pyplot
dev_pos = numpy.random.normal(1,1,100)
......@@ -13,6 +13,6 @@ dev_neg = numpy.random.normal(-1,1,100)
test_pos = numpy.random.normal(0.9,1,100)
test_neg = numpy.random.normal(-1.1,1,100)
npoints = 100
bob.measure.plot.epc(dev_neg, dev_pos, test_neg, test_pos, npoints, color=(0,0,0), linestyle='-')
xbob.measure.plot.epc(dev_neg, dev_pos, test_neg, test_pos, npoints, color=(0,0,0), linestyle='-')
pyplot.grid(True)
pyplot.title('EPC')
......@@ -4,14 +4,14 @@
"""Tutorial for plotting a ROC curve
"""
import bob
import numpy
import xbob.measure
from matplotlib import pyplot
positives = numpy.random.normal(1,1,100)
negatives = numpy.random.normal(-1,1,100)
npoints = 100
bob.measure.plot.roc(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test')
xbob.measure.plot.roc(negatives, positives, npoints, color=(0,0,0), linestyle='-', label='test')
pyplot.grid(True)
pyplot.xlabel('FAR (%)')
pyplot.ylabel('FRR (%)')
......
......@@ -6,8 +6,8 @@
Python API
============
This section includes information for using the pure Python API of ``xbob.io``.
This section includes information for using the pure Python API of ``bob::measure``.
.. automodule:: xbob.io
.. automodule:: xbob.measure
......@@ -4,16 +4,16 @@
# Wed Apr 20 17:32:54 2011 +0200
#
# Copyright (C) 2011-2013 Idiap Research Institute, Martigny, Switzerland
#
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
......@@ -22,7 +22,7 @@
For separability information, please consult: http://en.wikipedia.org/wiki/File:Anderson%27s_Iris_data_set.png
"""
import bob
import xbob.io
iris_columns = {
'sepal.length': 0,
......@@ -49,7 +49,7 @@ def loaddata(filename, column):
def example1():
"""In the first example we will get a linearly separable set of scores:
Variable: Petal length
Iris setosa: noise
Iris virginica: signal
......@@ -57,8 +57,8 @@ def example1():
Separation threshold is about 3.
"""
data = loaddata('iris.data', 'petal.length')
bob.io.save(data['setosa'], 'linsep-negatives.hdf5')
bob.io.save(data['virginica'],'linsep-positives.hdf5')
xbob.io.save(data['setosa'], 'linsep-negatives.hdf5')
xbob.io.save(data['virginica'],'linsep-positives.hdf5')
def example2():
"""In the second example we will get a non-linearly separable set of scores:
......@@ -70,8 +70,8 @@ def example2():
Separation threshold is about 5 (min. HTER).
"""
data = loaddata('iris.data', 'sepal.length')
bob.io.save(data['setosa'], 'nonsep-negatives.hdf5')
bob.io.save(data['versicolor'], 'nonsep-positives.hdf5')
xbob.io.save(data['setosa'], 'nonsep-negatives.hdf5')
xbob.io.save(data['versicolor'], 'nonsep-positives.hdf5')
def main():
"""Generates data for all examples."""
......
......@@ -231,26 +231,26 @@ def det(negatives, positives, npoints=100, axisfontsize='x-small', **kwargs):
If you wish to reset axis zooming, you must use the gaussian scale rather
than the visual marks showed at the plot, which are just there for
displaying purposes. The real axis scale is based on the
bob.measure.ppndf() method. For example, if you wish to set the x and y
xbob.measure.ppndf() method. For example, if you wish to set the x and y
axis to display data between 1% and 40% here is the recipe:
.. code-block:: python
import bob
import xbob.measure
import matplotlib.pyplot as mpl
bob.measure.plot.det(...) #call this as many times as you need
xbob.measure.plot.det(...) #call this as many times as you need
#AFTER you plot the DET curve, just set the axis in this way:
mpl.axis([bob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)])
mpl.axis([xbob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)])
We provide a convenient way for you to do the above in this module. So,
optionally, you may use the bob.measure.plot.det_axis() method like this:
optionally, you may use the xbob.measure.plot.det_axis() method like this:
.. code-block:: python
import bob
bob.measure.plot.det(...)
import xbob.measure
xbob.measure.plot.det(...)
# please note we convert percentage values in det_axis()
bob.measure.plot.det_axis([1, 40, 1, 40])
xbob.measure.plot.det_axis([1, 40, 1, 40])
Return value is the matplotlib line that was added as defined by the
matplotlib.pyplot.plot() command.
......@@ -308,8 +308,8 @@ def det_axis(v, **kwargs):
"""Sets the axis in a DET plot.
This method wraps the matplotlib.pyplot.axis() by calling
bob.measure.ppndf() on the values passed by the user so they are meaningful
in a DET plot as performed by bob.measure.plot.det().
xbob.measure.ppndf() on the values passed by the user so they are meaningful
in a DET plot as performed by xbob.measure.plot.det().
Keyword parameters:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment