Skip to content
Snippets Groups Projects
Commit 91b44bd9 authored by Theophile GENTILHOMME's avatar Theophile GENTILHOMME
Browse files

Fix doc and add guide description

parent 880d989b
No related branches found
No related tags found
2 merge requests!54Refactors the score loading and scripts functionality,!51Confidence intervals
Pipeline #
...@@ -9,24 +9,20 @@ def confidence_for_indicator_variable(x, n, alpha=0.05): ...@@ -9,24 +9,20 @@ def confidence_for_indicator_variable(x, n, alpha=0.05):
The Clopper-Pearson interval method is used for estimating the confidence The Clopper-Pearson interval method is used for estimating the confidence
intervals. intervals.
More info on confidence intervals
---------------------------------
https://en.wikipedia.org/wiki/Confidence_interval
https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Clopper-Pearson_interval
Parameters Parameters
---------- ----------
x : int x : int
The number of successes. The number of successes.
n : int n : int
The number of trials. The number of trials.
alpha : float, optional alpha : :obj:`float`, optional
The 1-confidence value that you want. For example, alpha should be 0.05 The 1-confidence value that you want. For example, alpha should be 0.05
to obtain 95% confidence intervals. to obtain 95% confidence intervals.
Returns Returns
------- -------
(float, float) Returns a tuple of (lower_bound, upper_bound) which (:obj:`float`, :obj:`float`)
a tuple of (lower_bound, upper_bound) which
shows the limit of your success rate: lower_bound < x/n < upper_bound shows the limit of your success rate: lower_bound < x/n < upper_bound
''' '''
lower_bound = scipy.stats.beta.ppf(alpha / 2.0, x, n - x + 1) lower_bound = scipy.stats.beta.ppf(alpha / 2.0, x, n - x + 1)
......
...@@ -205,6 +205,25 @@ Both functions require that at least one probe item exists, which has no accordi ...@@ -205,6 +205,25 @@ Both functions require that at least one probe item exists, which has no accordi
>>> dir = bob.measure.detection_identification_rate(rr_scores, threshold = 0, rank=1) >>> dir = bob.measure.detection_identification_rate(rr_scores, threshold = 0, rank=1)
>>> far = bob.measure.false_alarm_rate(rr_scores, threshold = 0) >>> far = bob.measure.false_alarm_rate(rr_scores, threshold = 0)
Confidence interval
-------------------
A confidence interval for parameter `x` consists of a lower
estimate `L`, and an upper estimate `U`, such that the probability of the true value being
within the interval estimate is equal to `\alpha`. For example,
a 95% confidence interval (i.e. `\alpha = 0.95`) for a parameter `x` is given by `[L, U]` such that
`Prob(x∈[L,U]) = 95%`. The smaller the test size, the wider the confidence
interval will be, and the greater `alpha`, the smaller the confidence interval
will be.
`The Clopper-Pearson interval`_, a common method for calculating
confidence intervals, is function of the number of success, the number of trials
and confidence
value `\alpha` is used as :py:func:`bob.measure.utils.confidence_for_indicator_variable`.
It is based on the cumulative probabilities of the binomial distribution. This
method is quite conservative, meaning that the true coverage rate of a 95%
Clopper–Pearson interval may be well above 95%.
Plotting Plotting
-------- --------
...@@ -436,4 +455,4 @@ that best suits you. ...@@ -436,4 +455,4 @@ that best suits you.
.. _`The Expected Performance Curve`: http://publications.idiap.ch/downloads/reports/2005/bengio_2005_icml.pdf .. _`The Expected Performance Curve`: http://publications.idiap.ch/downloads/reports/2005/bengio_2005_icml.pdf
.. _`The DET curve in assessment of detection task performance`: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117.4489&rep=rep1&type=pdf .. _`The DET curve in assessment of detection task performance`: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117.4489&rep=rep1&type=pdf
.. _openbr: http://openbiometrics.org .. _`The Clopper-Pearson interval`: https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Clopper-Pearson_interval
...@@ -64,6 +64,13 @@ Generic ...@@ -64,6 +64,13 @@ Generic
bob.measure.rmse bob.measure.rmse
bob.measure.get_config bob.measure.get_config
Confidence interval
-------------------
.. autosummary::
bob.measure.utils.confidence_for_indicator_variable
Calibration Calibration
----------- -----------
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment