Commit ec617147 authored by André Anjos's avatar André Anjos 💬

Merge branch 'eer' into 'master'

Explain that the thresholds might not satisfy the requested criteria

See merge request !69
parents cb3964bd b9c497b3
Pipeline #21007 passed with stages
in 29 minutes and 40 seconds
......@@ -168,7 +168,7 @@ def recognition_rate(cmc_scores, threshold = None, rank = 1):
Parameters:
cmc_scores (:py:class:`list`): A list in the format ``[(negatives,
positives), ...]`` containing the CMC scores (i.e. :py:class:`list`:
positives), ...]`` containing the CMC scores (i.e. :py:class:`list`:
A list of tuples, where each tuple contains the
``negative`` and ``positive`` scores for one probe of the database).
......@@ -395,7 +395,7 @@ def false_alarm_rate(cmc_scores, threshold):
Parameters:
cmc_scores (:py:class:`list`): A list in the format ``[(negatives,
positives), ...]`` containing the CMC scores (i.e. :py:class:`list`:
positives), ...]`` containing the CMC scores (i.e. :py:class:`list`:
A list of tuples, where each tuple contains the
``negative`` and ``positive`` scores for one probe of the database).
......@@ -431,6 +431,42 @@ def false_alarm_rate(cmc_scores, threshold):
return float(incorrect) / float(counter)
def eer(negatives, positives, is_sorted=False, also_farfrr=False):
"""Calculates the Equal Error Rate (EER).
Please note that it is possible that eer != far != frr.
This function returns (far + frr) / 2 as eer.
If you also need the far and frr values, set ``also_farfrr`` to ``True``.
Parameters
----------
negatives : ``array_like (1D, float)``
The scores for comparisons of objects of different classes.
positives : ``array_like (1D, float)``
The scores for comparisons of objects of the same class.
is_sorted : bool
Are both sets of scores already in ascendantly sorted order?
also_farfrr : bool
If True, it will also return far and frr.
Returns
-------
eer : float
The Equal Error Rate (EER).
far : float
The False Accept Rate (FAR). Returned only when ``also_farfrr`` is
``True``.
frr : float
The False Reject Rate (FAR). Returned only when ``also_farfrr`` is
``True``.
"""
threshold = eer_threshold(negatives, positives, is_sorted)
far, frr = farfrr(negatives, positives, threshold)
if also_farfrr:
return (far + frr) / 2.0, far, frr
return (far + frr) / 2.0
def get_config():
"""Returns a string containing the configuration information.
"""
......
......@@ -164,6 +164,20 @@ calculation of the threshold:
By setting cost to 0.5 is equivalent to use
:py:func:`bob.measure.min_hter_threshold`.
.. important::
Often, it is not numerically possible to match the requested criteria for
calculating the threshold based on the provided scores. Instead, the closest
possible threshold is returned. For example, using
:any:`bob.measure.eer_threshold` **will not** give you a threshold where
:math:`FAR == FRR`. Hence, you cannot report :math:`FAR` or :math:`FRR`
instead of :math:`EER`; you should report :math:`(FAR+FRR)/2` instead. This
is also true for :any:`bob.measure.far_threshold` and
:any:`bob.measure.frr_threshold`. The threshold returned by those functions
does not guarantee that using that threshold you will get the requested
:math:`FAR` or :math:`FRR` value. Instead, you should recalculate using
:any:`bob.measure.farfrr`.
.. note::
Many functions in ``bob.measure`` have an ``is_sorted`` parameter, which defaults to ``False``, throughout.
However, these functions need sorted ``positive`` and/or ``negative`` scores.
......
......@@ -23,6 +23,7 @@ Single point measurements
+++++++++++++++++++++++++
.. autosummary::
bob.measure.eer
bob.measure.farfrr
bob.measure.f_score
bob.measure.precision_recall
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment