plot.py 19.9 KB
 André Anjos committed Nov 21, 2013 1 2 3 4 ``````#!/usr/bin/env python # vim: set fileencoding=utf-8 : # Mon 23 May 2011 14:36:14 CEST `````` André Anjos committed Sep 28, 2016 5 `````` `````` Amir Mohammadi committed Apr 20, 2017 6 ``````def log_values(min_step=-4, counts_per_step=4): `````` André Anjos committed Sep 28, 2016 7 8 9 10 11 12 13 14 `````` """Computes log-scaled values between :math:`10^{M}` and 1 This function computes log-scaled values between :math:`10^{M}` and 1 (including), where :math:`M` is the ``min_ste`` argument, which needs to be a negative integer. The integral ``counts_per_step`` value defines how many values between two adjacent powers of 10 will be created. The total number of values will be ``-min_step * counts_per_step + 1``. `````` 15 `````` `````` André Anjos committed Sep 28, 2016 16 `````` Parameters: `````` 17 `````` `````` André Anjos committed Sep 29, 2016 18 19 20 `````` min_step (:py:class:`int`, optional): The power of 10 that will be the minimum value. E.g., the default ``-4`` will result in the first number to be :math:`10^{-4}` = ``0.00001`` or ``0.01%`` `````` 21 `````` `````` André Anjos committed Sep 29, 2016 22 23 24 25 `````` counts_per_step (:py:class:`int`, optional): The number of values that will be put between two adjacent powers of 10. With the default value ``4`` (and default values of ``min_step``), we will get ``log_list[0] == 1e-4``, ``log_list[4] == 1e-3``, ..., ``log_list[16] == 1``. `````` 26 27 `````` `````` André Anjos committed Sep 28, 2016 28 29 `````` Returns: `````` André Anjos committed May 22, 2017 30 31 `````` :py:class:`list`: A list of logarithmically scaled values between :math:`10^{M}` and 1. `````` 32 33 `````` """ `````` André Anjos committed Sep 28, 2016 34 `````` `````` 35 `````` import math `````` Amir Mohammadi committed Apr 20, 2017 36 `````` return [math.pow(10., i * 1. / counts_per_step) for i in range(min_step * counts_per_step, 0)] + [1.] `````` 37 `````` `````` André Anjos committed Nov 21, 2013 38 39 `````` def roc(negatives, positives, npoints=100, CAR=False, **kwargs): `````` 40 `````` """Plots Receiver Operating Characteristic (ROC) curve. `````` André Anjos committed Nov 21, 2013 41 `````` `````` Manuel Günther committed Nov 25, 2015 42 `````` This method will call ``matplotlib`` to plot the ROC curve for a system which `````` André Anjos committed Nov 21, 2013 43 `````` contains a particular set of negatives (impostors) and positives (clients) `````` André Anjos committed Sep 28, 2016 44 45 46 `````` scores. We use the standard :py:func:`matplotlib.pyplot.plot` command. All parameters passed with exception of the three first parameters of this method will be directly passed to the plot command. `````` André Anjos committed Nov 21, 2013 47 `````` `````` André Anjos committed Sep 28, 2016 48 49 50 `````` The plot will represent the false-alarm on the horizontal axis and the false-rejection on the vertical axis. The values for the axis will be computed using :py:func:`bob.measure.roc`. `````` André Anjos committed Nov 21, 2013 51 `````` `````` Manuel Günther committed Nov 25, 2015 52 `````` .. note:: `````` André Anjos committed Nov 21, 2013 53 `````` `````` Manuel Günther committed Nov 25, 2015 54 55 56 `````` This function does not initiate and save the figure instance, it only issues the plotting command. You are the responsible for setting up and saving the figure as you see fit. `````` André Anjos committed Nov 21, 2013 57 58 `````` `````` André Anjos committed Sep 28, 2016 59 `````` Parameters: `````` André Anjos committed Nov 21, 2013 60 `````` `````` André Anjos committed Sep 28, 2016 61 62 63 `````` negatives (array): 1D float array that contains the scores of the "negative" (noise, non-class) samples of your classifier. See (:py:func:`bob.measure.roc`) `````` André Anjos committed Nov 21, 2013 64 `````` `````` André Anjos committed Sep 28, 2016 65 66 67 `````` positives (array): 1D float array that contains the scores of the "positive" (signal, class) samples of your classifier. See (:py:func:`bob.measure.roc`) `````` André Anjos committed Nov 21, 2013 68 `````` `````` André Anjos committed Sep 29, 2016 69 `````` npoints (:py:class:`int`, optional): The number of points for the plot. See `````` André Anjos committed Sep 28, 2016 70 `````` (:py:func:`bob.measure.roc`) `````` André Anjos committed Nov 21, 2013 71 `````` `````` André Anjos committed Sep 29, 2016 72 73 74 `````` CAR (:py:class:`bool`, optional): If set to ``True``, it will plot the CAR over FAR in using :py:func:`matplotlib.pyplot.semilogx`, otherwise the FAR over FRR linearly using :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Sep 28, 2016 75 `````` `````` André Anjos committed Sep 29, 2016 76 77 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Sep 28, 2016 78 79 80 81 `````` Returns: `````` André Anjos committed Sep 29, 2016 82 83 84 `````` :py:class:`list` of :py:class:`matplotlib.lines.Line2D`: The lines that were added as defined by the return value of :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Nov 21, 2013 85 86 87 `````` """ `````` Manuel Günther committed Nov 25, 2015 88 `````` from matplotlib import pyplot `````` André Anjos committed Nov 21, 2013 89 `````` from . import roc as calc `````` André Anjos committed Dec 12, 2013 90 `````` out = calc(negatives, positives, npoints) `````` André Anjos committed Nov 21, 2013 91 `````` if not CAR: `````` Amir Mohammadi committed Apr 20, 2017 92 `````` return pyplot.plot(100.0 * out[0, :], 100.0 * out[1, :], **kwargs) `````` André Anjos committed Nov 21, 2013 93 `````` else: `````` Amir Mohammadi committed Apr 20, 2017 94 `````` return pyplot.semilogx(100.0 * out[0, :], 100.0 * (1 - out[1, :]), **kwargs) `````` Manuel Günther committed Nov 25, 2015 95 `````` `````` André Anjos committed Nov 21, 2013 96 `````` `````` Amir Mohammadi committed Apr 20, 2017 97 ``````def roc_for_far(negatives, positives, far_values=log_values(), **kwargs): `````` André Anjos committed Sep 28, 2016 98 `````` """Plots the ROC curve for the given list of False Acceptance Rates (FAR). `````` 99 100 101 `````` This method will call ``matplotlib`` to plot the ROC curve for a system which contains a particular set of negatives (impostors) and positives (clients) `````` André Anjos committed Sep 28, 2016 102 103 104 `````` scores. We use the standard :py:func:`matplotlib.pyplot.semilogx` command. All parameters passed with exception of the three first parameters of this method will be directly passed to the plot command. `````` 105 `````` `````` André Anjos committed Sep 28, 2016 106 107 108 `````` The plot will represent the False Acceptance Rate (FAR) on the horizontal axis and the Correct Acceptance Rate (CAR) on the vertical axis. The values for the axis will be computed using :py:func:`bob.measure.roc_for_far`. `````` 109 110 111 112 113 114 115 116 `````` .. note:: This function does not initiate and save the figure instance, it only issues the plotting command. You are the responsible for setting up and saving the figure as you see fit. `````` André Anjos committed Sep 28, 2016 117 118 119 120 121 `````` Parameters: negatives (array): 1D float array that contains the scores of the "negative" (noise, non-class) samples of your classifier. See (:py:func:`bob.measure.roc`) `````` 122 `````` `````` André Anjos committed Sep 28, 2016 123 124 125 `````` positives (array): 1D float array that contains the scores of the "positive" (signal, class) samples of your classifier. See (:py:func:`bob.measure.roc`) `````` 126 `````` `````` André Anjos committed Sep 29, 2016 127 128 `````` far_values (:py:class:`list`, optional): The values for the FAR, where the CAR should be plotted; each value should be in range [0,1]. `````` 129 `````` `````` André Anjos committed Sep 29, 2016 130 131 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Sep 28, 2016 132 133 134 135 `````` Returns: `````` André Anjos committed Sep 29, 2016 136 137 138 `````` :py:class:`list` of :py:class:`matplotlib.lines.Line2D`: The lines that were added as defined by the return value of :py:func:`matplotlib.pyplot.semilogx`. `````` 139 140 141 142 143 144 `````` """ from matplotlib import pyplot from . import roc_for_far as calc out = calc(negatives, positives, far_values) `````` Amir Mohammadi committed Apr 20, 2017 145 `````` return pyplot.semilogx(100.0 * out[0, :], 100.0 * (1 - out[1, :]), **kwargs) `````` 146 147 `````` `````` André Anjos committed Nov 21, 2013 148 ``````def precision_recall_curve(negatives, positives, npoints=100, **kwargs): `````` André Anjos committed Sep 28, 2016 149 `````` """Plots a Precision-Recall curve. `````` André Anjos committed Nov 21, 2013 150 `````` `````` André Anjos committed Sep 28, 2016 151 152 153 154 155 156 `````` This method will call ``matplotlib`` to plot the precision-recall curve for a system which contains a particular set of ``negatives`` (impostors) and ``positives`` (clients) scores. We use the standard :py:func:`matplotlib.pyplot.plot` command. All parameters passed with exception of the three first parameters of this method will be directly passed to the plot command. `````` André Anjos committed Nov 21, 2013 157 `````` `````` Manuel Günther committed Nov 25, 2015 158 `````` .. note:: `````` André Anjos committed Nov 21, 2013 159 `````` `````` Manuel Günther committed Nov 25, 2015 160 161 162 `````` This function does not initiate and save the figure instance, it only issues the plotting command. You are the responsible for setting up and saving the figure as you see fit. `````` André Anjos committed Nov 21, 2013 163 164 `````` `````` André Anjos committed Sep 28, 2016 165 166 167 168 169 170 171 172 173 `````` Parameters: negatives (array): 1D float array that contains the scores of the "negative" (noise, non-class) samples of your classifier. See (:py:func:`bob.measure.precision_recall_curve`) positives (array): 1D float array that contains the scores of the "positive" (signal, class) samples of your classifier. See (:py:func:`bob.measure.precision_recall_curve`) `````` André Anjos committed Nov 21, 2013 174 `````` `````` André Anjos committed Sep 29, 2016 175 `````` npoints (:py:class:`int`, optional): The number of points for the plot. See `````` André Anjos committed Sep 28, 2016 176 `````` (:py:func:`bob.measure.precision_recall_curve`) `````` André Anjos committed Nov 21, 2013 177 `````` `````` André Anjos committed Sep 29, 2016 178 179 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Nov 21, 2013 180 181 `````` `````` André Anjos committed Sep 28, 2016 182 183 `````` Returns: `````` André Anjos committed Sep 29, 2016 184 185 186 `````` :py:class:`list` of :py:class:`matplotlib.lines.Line2D`: The lines that were added as defined by the return value of :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Sep 28, 2016 187 `````` `````` André Anjos committed Nov 21, 2013 188 189 `````` """ `````` Manuel Günther committed Nov 25, 2015 190 `````` from matplotlib import pyplot `````` André Anjos committed Nov 21, 2013 191 192 `````` from . import precision_recall_curve as calc out = calc(negatives, positives, npoints) `````` Amir Mohammadi committed Apr 20, 2017 193 `````` return pyplot.plot(100.0 * out[0, :], 100.0 * out[1, :], **kwargs) `````` André Anjos committed Nov 21, 2013 194 195 196 `````` def epc(dev_negatives, dev_positives, test_negatives, test_positives, `````` Amir Mohammadi committed Apr 20, 2017 197 `````` npoints=100, **kwargs): `````` André Anjos committed Nov 21, 2013 198 199 200 201 202 203 204 `````` """Plots Expected Performance Curve (EPC) as defined in the paper: Bengio, S., Keller, M., Mariéthoz, J. (2004). The Expected Performance Curve. International Conference on Machine Learning ICML Workshop on ROC Analysis in Machine Learning, 136(1), 1963–1966. IDIAP RR. Available: http://eprints.pascal-network.org/archive/00000670/ `````` Manuel Günther committed Nov 25, 2015 205 `````` This method will call ``matplotlib`` to plot the EPC curve for a system which `````` André Anjos committed Nov 21, 2013 206 207 `````` contains a particular set of negatives (impostors) and positives (clients) for both the development and test sets. We use the standard `````` André Anjos committed Sep 28, 2016 208 209 210 `````` :py:func:`matplotlib.pyplot.plot` command. All parameters passed with exception of the five first parameters of this method will be directly passed to the plot command. `````` André Anjos committed Nov 21, 2013 211 `````` `````` André Anjos committed Sep 28, 2016 212 213 `````` The plot will represent the minimum HTER on the vertical axis and the cost on the horizontal axis. `````` André Anjos committed Nov 21, 2013 214 `````` `````` Manuel Günther committed Nov 25, 2015 215 `````` .. note:: `````` André Anjos committed Nov 21, 2013 216 `````` `````` Manuel Günther committed Nov 25, 2015 217 218 219 `````` This function does not initiate and save the figure instance, it only issues the plotting commands. You are the responsible for setting up and saving the figure as you see fit. `````` André Anjos committed Nov 21, 2013 220 221 `````` `````` André Anjos committed Sep 28, 2016 222 223 224 225 226 227 228 229 230 `````` Parameters: dev_negatives (array): 1D float array that contains the scores of the "negative" (noise, non-class) samples of your classifier, from the development set. See (:py:func:`bob.measure.epc`) dev_positives (array): 1D float array that contains the scores of the "positive" (signal, class) samples of your classifier, from the development set. See (:py:func:`bob.measure.epc`) `````` André Anjos committed Nov 21, 2013 231 `````` `````` André Anjos committed Sep 28, 2016 232 233 234 `````` test_negatives (array): 1D float array that contains the scores of the "negative" (noise, non-class) samples of your classifier, from the test set. See (:py:func:`bob.measure.epc`) `````` André Anjos committed Nov 21, 2013 235 `````` `````` André Anjos committed Sep 28, 2016 236 237 238 `````` test_positives (array): 1D float array that contains the scores of the "positive" (signal, class) samples of your classifier, from the test set. See (:py:func:`bob.measure.epc`) `````` André Anjos committed Nov 21, 2013 239 `````` `````` André Anjos committed Sep 29, 2016 240 `````` npoints (:py:class:`int`, optional): The number of points for the plot. See `````` André Anjos committed Sep 28, 2016 241 242 `````` (:py:func:`bob.measure.epc`) `````` André Anjos committed Sep 29, 2016 243 244 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Sep 28, 2016 245 246 247 248 `````` Returns: `````` André Anjos committed Sep 29, 2016 249 250 251 `````` :py:class:`list` of :py:class:`matplotlib.lines.Line2D`: The lines that were added as defined by the return value of :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Nov 21, 2013 252 253 254 `````` """ `````` Manuel Günther committed Nov 25, 2015 255 `````` from matplotlib import pyplot `````` André Anjos committed Nov 21, 2013 256 257 `````` from . import epc as calc `````` André Anjos committed Sep 28, 2016 258 `````` out = calc(dev_negatives, dev_positives, test_negatives, test_positives, `````` Amir Mohammadi committed Apr 20, 2017 259 260 `````` npoints) return pyplot.plot(out[0, :], 100.0 * out[1, :], **kwargs) `````` Manuel Günther committed Nov 25, 2015 261 `````` `````` André Anjos committed Nov 21, 2013 262 263 264 265 266 267 268 269 270 `````` def det(negatives, positives, npoints=100, axisfontsize='x-small', **kwargs): """Plots Detection Error Trade-off (DET) curve as defined in the paper: Martin, A., Doddington, G., Kamm, T., Ordowski, M., & Przybocki, M. (1997). The DET curve in assessment of detection task performance. Fifth European Conference on Speech Communication and Technology (pp. 1895-1898). Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117.4489&rep=rep1&type=pdf `````` André Anjos committed Sep 28, 2016 271 272 273 274 275 `````` This method will call ``matplotlib`` to plot the DET curve(s) for a system which contains a particular set of negatives (impostors) and positives (clients) scores. We use the standard :py:func:`matplotlib.pyplot.plot` command. All parameters passed with exception of the three first parameters of this method will be directly passed to the plot command. `````` André Anjos committed Nov 21, 2013 276 `````` `````` Manuel Günther committed Nov 25, 2015 277 278 `````` The plot will represent the false-alarm on the horizontal axis and the false-rejection on the vertical axis. `````` André Anjos committed Nov 21, 2013 279 280 281 282 283 284 285 286 287 288 289 290 291 292 `````` This method is strongly inspired by the NIST implementation for Matlab, called DETware, version 2.1 and available for download at the NIST website: http://www.itl.nist.gov/iad/mig/tools/ .. note:: This function does not initiate and save the figure instance, it only issues the plotting commands. You are the responsible for setting up and saving the figure as you see fit. .. note:: `````` Manuel Günther committed Nov 25, 2015 293 `````` If you wish to reset axis zooming, you must use the Gaussian scale rather `````` André Anjos committed Nov 21, 2013 294 `````` than the visual marks showed at the plot, which are just there for `````` André Anjos committed Sep 28, 2016 295 296 297 `````` displaying purposes. The real axis scale is based on :py:func:`bob.measure.ppndf`. For example, if you wish to set the x and y axis to display data between 1% and 40% here is the recipe: `````` André Anjos committed Nov 21, 2013 298 299 300 `````` .. code-block:: python `````` André Anjos committed May 26, 2014 301 `````` import bob.measure `````` Manuel Günther committed Nov 25, 2015 302 `````` from matplotlib import pyplot `````` André Anjos committed May 26, 2014 303 `````` bob.measure.plot.det(...) #call this as many times as you need `````` André Anjos committed Nov 21, 2013 304 `````` #AFTER you plot the DET curve, just set the axis in this way: `````` Manuel Günther committed Nov 25, 2015 305 `````` pyplot.axis([bob.measure.ppndf(k/100.0) for k in (1, 40, 1, 40)]) `````` André Anjos committed Nov 21, 2013 306 307 `````` We provide a convenient way for you to do the above in this module. So, `````` André Anjos committed Sep 28, 2016 308 309 `````` optionally, you may use the :py:func:`bob.measure.plot.det_axis` method like this: `````` André Anjos committed Nov 21, 2013 310 311 312 `````` .. code-block:: python `````` André Anjos committed May 26, 2014 313 314 `````` import bob.measure bob.measure.plot.det(...) `````` André Anjos committed Nov 21, 2013 315 `````` # please note we convert percentage values in det_axis() `````` André Anjos committed May 26, 2014 316 `````` bob.measure.plot.det_axis([1, 40, 1, 40]) `````` André Anjos committed Nov 21, 2013 317 318 `````` `````` André Anjos committed Sep 28, 2016 319 320 321 322 323 `````` Parameters: negatives (array): 1D float array that contains the scores of the "negative" (noise, non-class) samples of your classifier. See (:py:func:`bob.measure.det`) `````` Manuel Günther committed Nov 25, 2015 324 `````` `````` André Anjos committed Sep 28, 2016 325 326 327 `````` positives (array): 1D float array that contains the scores of the "positive" (signal, class) samples of your classifier. See (:py:func:`bob.measure.det`) `````` Manuel Günther committed Nov 25, 2015 328 `````` `````` André Anjos committed Sep 29, 2016 329 `````` npoints (:py:class:`int`, optional): The number of points for the plot. See `````` André Anjos committed Sep 28, 2016 330 `````` (:py:func:`bob.measure.det`) `````` Manuel Günther committed Nov 25, 2015 331 `````` `````` André Anjos committed Sep 29, 2016 332 333 `````` axisfontsize (:py:class:`str`, optional): The size to be used by x/y-tick-labels to set the font size on the axis `````` Manuel Günther committed Nov 25, 2015 334 `````` `````` André Anjos committed Sep 29, 2016 335 336 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Sep 28, 2016 337 338 339 340 `````` Returns: `````` André Anjos committed Sep 29, 2016 341 342 343 `````` :py:class:`list` of :py:class:`matplotlib.lines.Line2D`: The lines that were added as defined by the return value of :py:func:`matplotlib.pyplot.plot`. `````` Manuel Günther committed Nov 25, 2015 344 345 `````` """ `````` André Anjos committed Nov 21, 2013 346 347 348 349 350 351 352 353 354 355 356 357 `````` # these are some constants required in this method desiredTicks = [ "0.00001", "0.00002", "0.00005", "0.0001", "0.0002", "0.0005", "0.001", "0.002", "0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.4", "0.6", "0.8", "0.9", "0.95", "0.98", "0.99", "0.995", "0.998", "0.999", "0.9995", "0.9998", "0.9999", "0.99995", "0.99998", "0.99999" `````` Amir Mohammadi committed Apr 20, 2017 358 `````` ] `````` André Anjos committed Nov 21, 2013 359 360 361 362 363 364 365 366 367 368 369 `````` desiredLabels = [ "0.001", "0.002", "0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1", "2", "5", "10", "20", "40", "60", "80", "90", "95", "98", "99", "99.5", "99.8", "99.9", "99.95", "99.98", "99.99", "99.995", "99.998", "99.999" `````` Amir Mohammadi committed Apr 20, 2017 370 `````` ] `````` André Anjos committed Nov 21, 2013 371 372 `````` # this will actually do the plotting `````` Manuel Günther committed Nov 25, 2015 373 `````` from matplotlib import pyplot `````` André Anjos committed Nov 21, 2013 374 375 376 `````` from . import det as calc from . import ppndf `````` André Anjos committed Dec 12, 2013 377 `````` out = calc(negatives, positives, npoints) `````` Amir Mohammadi committed Apr 20, 2017 378 `````` retval = pyplot.plot(out[0, :], out[1, :], **kwargs) `````` André Anjos committed Nov 21, 2013 379 380 381 `````` # now the trick: we must plot the tick marks by hand using the PPNDF method pticks = [ppndf(float(v)) for v in desiredTicks] `````` Amir Mohammadi committed Apr 20, 2017 382 `````` ax = pyplot.gca() # and finally we set our own tick marks `````` André Anjos committed Nov 21, 2013 383 384 385 386 387 388 389 `````` ax.set_xticks(pticks) ax.set_xticklabels(desiredLabels, size=axisfontsize) ax.set_yticks(pticks) ax.set_yticklabels(desiredLabels, size=axisfontsize) return retval `````` Manuel Günther committed Nov 25, 2015 390 `````` `````` André Anjos committed Nov 21, 2013 391 392 393 ``````def det_axis(v, **kwargs): """Sets the axis in a DET plot. `````` Manuel Günther committed Nov 25, 2015 394 `````` This method wraps the :py:func:`matplotlib.pyplot.axis` by calling `````` André Anjos committed Sep 28, 2016 395 396 397 398 399 `````` :py:func:`bob.measure.ppndf` on the values passed by the user so they are meaningful in a DET plot as performed by :py:func:`bob.measure.plot.det`. Parameters: `````` Manuel Günther committed Nov 25, 2015 400 `````` `````` André Anjos committed Sep 29, 2016 401 402 403 404 405 `````` v (``sequence``): A sequence (list, tuple, array or the like) containing the X and Y limits in the order ``(xmin, xmax, ymin, ymax)``. Expected values should be in percentage (between 0 and 100%). If ``v`` is not a list or tuple that contains 4 numbers it is passed without further inspection to :py:func:`matplotlib.pyplot.axis`. `````` André Anjos committed Nov 21, 2013 406 `````` `````` André Anjos committed Sep 29, 2016 407 408 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.axis`. `````` André Anjos committed Nov 21, 2013 409 410 `````` `````` André Anjos committed Sep 28, 2016 411 412 413 `````` Returns: object: Whatever is returned by :py:func:`matplotlib.pyplot.axis`. `````` André Anjos committed Nov 21, 2013 414 415 416 417 `````` """ import logging `````` Manuel Günther committed Nov 25, 2015 418 `````` logger = logging.getLogger("bob.measure") `````` André Anjos committed Nov 21, 2013 419 `````` `````` Manuel Günther committed Nov 25, 2015 420 `````` from matplotlib import pyplot `````` André Anjos committed Nov 21, 2013 421 422 423 424 `````` from . import ppndf # treat input try: `````` Amir Mohammadi committed Apr 20, 2017 425 426 427 428 `````` tv = list(v) # normal input if len(tv) != 4: raise IndexError tv = [ppndf(float(k) / 100) for k in tv] `````` Manuel Günther committed Nov 25, 2015 429 `````` cur = pyplot.axis() `````` André Anjos committed Nov 21, 2013 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 `````` # limits must be within bounds if tv[0] < cur[0]: logger.warn("Readjusting xmin: the provided value is out of bounds") tv[0] = cur[0] if tv[1] > cur[1]: logger.warn("Readjusting xmax: the provided value is out of bounds") tv[1] = cur[1] if tv[2] < cur[2]: logger.warn("Readjusting ymin: the provided value is out of bounds") tv[2] = cur[2] if tv[3] > cur[3]: logger.warn("Readjusting ymax: the provided value is out of bounds") tv[3] = cur[3] except: tv = v `````` Manuel Günther committed Nov 25, 2015 448 449 `````` return pyplot.axis(tv, **kwargs) `````` André Anjos committed Nov 21, 2013 450 `````` `````` Amir Mohammadi committed Apr 20, 2017 451 ``````def cmc(cmc_scores, logx=True, **kwargs): `````` André Anjos committed Sep 28, 2016 452 453 454 455 456 457 458 459 460 461 `````` """Plots the (cumulative) match characteristics and returns the maximum rank. This function plots a CMC curve using the given CMC scores, which can be read from the our score files using the :py:func:`bob.measure.load.cmc_four_column` or :py:func:`bob.measure.load.cmc_five_column` methods. The structure of the ``cmc_scores`` parameter is relatively complex. It contains a list of pairs of lists. For each probe object, a pair of list negative and positive scores is required. `````` Manuel Günther committed Sep 17, 2015 462 `````` `````` André Anjos committed Sep 28, 2016 463 `````` Parameters: `````` Manuel Günther committed Sep 17, 2015 464 `````` `````` André Anjos committed Sep 28, 2016 465 466 `````` cmc_scores (array): 1D float array containing the CMC values (See :py:func:`bob.measure.cmc`) `````` Manuel Günther committed Sep 17, 2015 467 `````` `````` André Anjos committed Sep 29, 2016 468 469 470 `````` logx (:py:class:`bool`, optional): If set (the default), plots the rank axis in logarithmic scale using :py:func:`matplotlib.pyplot.semilogx` or in linear scale using :py:func:`matplotlib.pyplot.plot` `````` Manuel Günther committed Sep 17, 2015 471 `````` `````` André Anjos committed Sep 29, 2016 472 473 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.plot`. `````` Manuel Günther committed Sep 17, 2015 474 `````` `````` Manuel Günther committed Nov 25, 2015 475 `````` `````` André Anjos committed Sep 28, 2016 476 477 478 `````` Returns: int: The number of classes (clients) in the given scores. `````` André Anjos committed Nov 21, 2013 479 `````` `````` Manuel Günther committed Nov 25, 2015 480 `````` """ `````` André Anjos committed Sep 28, 2016 481 `````` `````` Manuel Günther committed Nov 25, 2015 482 `````` from matplotlib import pyplot `````` André Anjos committed Nov 21, 2013 483 `````` from . import cmc as calc `````` Manuel Günther committed Nov 25, 2015 484 `````` `````` André Anjos committed Nov 21, 2013 485 486 487 `````` out = calc(cmc_scores) if logx: `````` Amir Mohammadi committed Apr 20, 2017 488 `````` pyplot.semilogx(range(1, len(out) + 1), out * 100, **kwargs) `````` André Anjos committed Nov 21, 2013 489 `````` else: `````` Amir Mohammadi committed Apr 20, 2017 490 `````` pyplot.plot(range(1, len(out) + 1), out * 100, **kwargs) `````` André Anjos committed Nov 21, 2013 491 492 `````` return len(out) `````` 493 494 `````` `````` Amir Mohammadi committed Apr 20, 2017 495 ``````def detection_identification_curve(cmc_scores, far_values=log_values(), rank=1, logx=True, **kwargs): `````` André Anjos committed Sep 28, 2016 496 `````` """Plots the Detection & Identification curve over the FAR `````` 497 `````` `````` André Anjos committed Sep 28, 2016 498 499 500 501 `````` This curve is designed to be used in an open set identification protocol, and defined in Chapter 14.1 of [LiJain2005]_. It requires to have at least one open set probe item, i.e., with no corresponding gallery, such that the positives for that pair are ``None``. `````` 502 `````` `````` André Anjos committed Sep 28, 2016 503 504 505 506 507 `````` The detection and identification curve first computes FAR thresholds based on the out-of-set probe scores (negative scores). For each probe item, the **maximum** negative score is used. Then, it plots the detection and identification rates for those thresholds, which are based on the in-set probe scores only. See [LiJain2005]_ for more details. `````` 508 `````` `````` André Anjos committed Sep 28, 2016 509 `````` .. [LiJain2005] **Stan Li and Anil K. Jain**, *Handbook of Face Recognition*, Springer, 2005 `````` 510 511 `````` `````` André Anjos committed Sep 28, 2016 512 `````` Parameters: `````` 513 `````` `````` André Anjos committed Sep 28, 2016 514 515 `````` cmc_scores (array): 1D float array containing the CMC values (See :py:func:`bob.measure.cmc`) `````` 516 `````` `````` André Anjos committed Sep 29, 2016 517 518 `````` rank (:py:class:`int`, optional): The rank for which the curve should be plotted `````` 519 `````` `````` André Anjos committed Sep 29, 2016 520 521 `````` far_values (:py:class:`list`, optional): The values for the FAR, where the CAR should be plotted; each value should be in range [0,1]. `````` 522 `````` `````` André Anjos committed Sep 29, 2016 523 524 525 `````` logx (:py:class:`bool`, optional): If set (the default), plots the rank axis in logarithmic scale using :py:func:`matplotlib.pyplot.semilogx` or in linear scale using :py:func:`matplotlib.pyplot.plot` `````` André Anjos committed Sep 28, 2016 526 `````` `````` André Anjos committed Sep 29, 2016 527 528 `````` kwargs (:py:class:`dict`, optional): Extra plotting parameters, which are passed directly to :py:func:`matplotlib.pyplot.plot`. `````` André Anjos committed Sep 28, 2016 529 530 531 532 `````` Returns: `````` André Anjos committed Sep 29, 2016 533 534 535 `````` :py:class:`list` of :py:class:`matplotlib.lines.Line2D`: The lines that were added as defined by the return value of :py:func:`matplotlib.pyplot.plot`. `````` 536 537 538 `````` """ `````` 539 `````` import numpy `````` Manuel Günther committed Aug 31, 2017 540 `````` import math `````` 541 `````` from matplotlib import pyplot `````` 542 `````` from . import far_threshold, detection_identification_rate `````` 543 `````` `````` Amir Mohammadi committed Apr 20, 2017 544 545 546 547 `````` # for each probe, for which no positives exists, get the highest negative # score; and sort them to compute the FAR thresholds negatives = sorted(max(neg) for neg, pos in cmc_scores if ( pos is None or not numpy.array(pos).size) and neg is not None) `````` 548 `````` if not negatives: `````` Amir Mohammadi committed Apr 20, 2017 549 550 `````` raise ValueError( "There need to be at least one pair with only negative scores") `````` 551 552 553 554 `````` # compute thresholds based on FAR values thresholds = [far_threshold(negatives, [], v, True) for v in far_values] `````` Amir Mohammadi committed Apr 20, 2017 555 556 `````` # compute detection and identification rate based on the thresholds for # the given rank `````` Manuel Günther committed Aug 31, 2017 557 558 `````` rates = [detection_identification_rate(cmc_scores, t, rank) if not math.isnan(t) else numpy.nan for t in thresholds] `````` 559 560 561 562 563 564 `````` # plot curve if logx: return pyplot.semilogx(far_values, rates, **kwargs) else: return pyplot.plot(far_values, rates, **kwargs)``````