• !100 Python-only version of bob.measure: This MR implements the following changes to this library, in an API backward-compatible way: 1. Minimize the number of Bob dependencies to a bare minimum. Use h5py to read data, RNGs and linear algebra from numpy. Remove requirements for bob.core, bob.io.base, bob.blitz, and most of bob.math (except for the PAVX algorithm) 2. Remove deprecated dependencies (docopt) 3. Use pytest for tests instead of nose 4. Re-implement the old C++ as a Python-only library (remove all C++ dependencies such as Boost)

• !99 Adjusted the histogram scale of the x axis to automatically adjust for varying scales across dev and eval

• !97 Add Area Under ROC Curve (AUC): Add the function called bob.measure.roc_auc_score. Add the AUC computation to bob measure metrics command. Fixes #2
• !98 Separate semilogx and TPR options in ROC plots: Also, ran black on some code files.

• !95 Improve threshold calculation in ROC, DET, and Precision and Recall plots: Fixes #58 and #60
• !96 Add an alpha option to ROC,DET,EPC figures: Also, enables you to change the default value in tablefmt option.
• Python 3.7 support

• !89 Add a base class for subplot-based plots
• !90 Fixing histogram test
• !91 Fix decimal option in metrics command: Fixes #52
• !43 Resolve "FAR and FRR thresholds are computed even when there is no data support"
• !92 Fix minimizing threshold: Closes #59
• !93 Click scripts fixes: Improvements and bug fixes to click scripts
• !94 Fix --figsize help message

• Breaking and significant changes:
• Removed the old plotting scripts (compute_perf.py, etc.).
• Added a new 2-column score format specific to bob.measure.
• Implemented generic metrics and plotting scripts. Do bob measure --help to see the new scripts and refer to the documentation.
• Matplotlib 2.2 and above is required now.
• Some biometric-related-only functionality is moved to bob.bio.base.
• Detailed changes:
• !52 generic plotting script for bob measure: From #37, provide generic plotting scripts: * bob measure evaluate * bob measure hist * bob measure hter * ...
• !55 Change option name criter to criterion
• !56 Fix error in context name
• !57 Extend bins number option
• !58 Change variable name form criter to criterion
• !59 Bug fix: incorrect input file reading
• !60 Bugfix
• !54 Refactors the score loading and scripts functionality: This merge request: * Move the biometric related functionality of bob.measure to bob.bio.base. * Add confidence interval calculations * Provide a score format for bob.measure with load functionalities * Provide a generic plotting script bob measure in bob.measure using bob.measure input file format: * bob measure metrics: to compute thresholds and evaluate performances * bob measure roc: to plot ROC * bob measure det: to plot DET * bob measure hist: to plot histograms * bob measure epc : to plot EPC * bob measure evaluate: applies all the above commands at once * bob measure gen: to generate fake scores for bob.measure Each command accepts one or several (dev,eval) score(s) file(s) for each system.
• !61 Title and histograms subplots: Account for this bob.bio.base!146#note_28760.

• Titles can be remove using an empty string, i.e. -t ' ' * Histograms support subplot display, see options --subplots * Add --legends-ncol option

• !62 Improve legends in histograms
• !48 recompute far values in roc_for_far: Fixes #27
• !44 Compute roc using roc_for_far internally: Fixes #26
• !43 Resolve "FAR and FRR thresholds are computed even when there is no data support": Closes #27 Also changes behavior of far_threshold and frr_threshold where the returned threshold guarantees the at most the requested far/frr value.
• !63 Enable semilogx option in roc curves: * Fixes #40 * Remove 0 points on x-axis in semilogx plots. Matplotlib 2 was doing this automatically before but matplotlib 2.2 doesn't * Code clean-up
• !64 Fix semilog plots for non numpy arrays: Fixes bob.bio.base#117
• !65 Modification of criterion_option: Correct display of available criteria
• !66 Add prefix aliasing: Prefix aliasing using AliasedGroup for bob.extension
• !68 Change --eval option default and Various fixes: Change defaults, clean unused options for histograms. Fix bob.bio.base#112
• !69 Explain that the thresholds might not satisfy the requested criteria: * Add a helper eer function for convenience.
• !50 Generic loading input file: Add a generic input file loading function for bob measure and its corresponding test. From issue #39, comments #39#note_26366.
• !71 Fix issues with matplotlib.hist.: For some reason mpl.hist hangs when n_bins is set to auto. This is related with https://github.com/numpy/numpy#8203 This MR set the default value to doane. Furthermore, allows you to pick one of the options here https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html#numpy.histogram This will fix one of the points here bob.pad.base!43
• !72 Various improvements: Fix #41 and #42
• !70 Improvements to new metrics codes
• !73 Histogram legends: Fix #44
• !74 Bins histograms: Option only depends on the number of data plotter per histograms, not the number of system. For example, pad hist requires 2 nbins and vuln hist requires 3 independently on the number of systems to be plotted. Fix #45.
• !75 Fix issue with histo legends: Fix #47 Also change default for number of legend columns. Add comments and doc in the code for Hist.
• !76 Histo fix: Fix typo and bob.pad.base!43#note_31884
• !77 dd lines for dev histograms: See bob.pad.base!43#note_31903
• !78 Metrics: All stuff related to #46.
• !79 Add a command for multi protocol (N-fold cross validation) analysis
• !80 Consider non 1 values negatives: This will make bob.measure scripts work with FOFRA scores
• !81 Compute HTER using FMR and FNMR: As discussed in the meeting. Fixes #48
• !83 Enable grid on all histograms
• !84 Improve the constrained layout option
• !82 Various fixes: * Change measure metrics * Document and change HTER * add decimal precision option for metric * Use acronyms instead of full names in figures * Remove filenames form figures and add log output instead
• !86 Fix decimal number control for metrics: Fixes #52
• !88 Fix tests: Should fix bob.nightlies#40
• !67 Titles: Allow list of titles and remove (development) (evaluation) when default titles are modified
• !45 Condapackage
• !85 Fix broken commands cref bob.extension!86
• !53 Change the way the scores arguments are passed to the compute() function: it now: Change the way the scores arguments are passed to the compute() function: it now does not rely on dev,eval pairs anymore and can take any number of different files (e.g. train)
• !87 Update documentation and commands: FAR->FPR, FRR->FNR: Fix #54

• Fixed issue with FAR and FRR values being exactly 1 or 0
• Corrected the threshold calculation in far_threshold() and frr_threshold(), which now guarantees to produce at most of given FAR or FRR values.
• Never return NaNs for far_threshold() and frr_threshold() functions
• Optimized implementation of ROC for a given FAR
• Added an option to capture the output in a log file
• Updated docs and tests accordingly

• Correctly handled NaN values in DIR plot
• Correctly handled of FAR and FRR thresholds
• Updated CMC documentation
• Switched to Numpy style documentation
• Fixed the dependency with bob.math (>=3.0.0)
• Fixed FAR and FRR threshold computation and return NaN when threshold cannot be computed

• !28 : Add an option to the epc function to also get the thresholds which were used during calculation

• Change the style of function comments
• Remove nan from scores and report FTA instead Report proper FMR, FNMR, FAR, FRR values
• Added min_weighted_error_rate_threshold in test for raised exceptions
• Implemented check on empty arrays for some functions -> raises excpetions now
• Used svg build badges
• Updated sphinx docs
• Load score more efficiently for negatives and positives
• Changed the ROC curve to implement what is required by 'The Handbook of Face Recognition'

This release brings some minor improvements to version 2.2.0 of this package:

• The use of a test-set on compute_perf is now optional
• Documentation concerning the scripts has been fixed to reflect changes on command-lines
• Implements bob.core verbosity controls on all scripts
• Fixes CMC plotting
• Re-organized score-loading
• Improved score-loading speeds

This release changes the command-line of most scripts in this package. Be attentive.

• Use docopt instead of argparse
• Make script interface more intuitive (remove non-optional options, use arguments)
• Fix all documentation warnings (nit-picky sphinx mode)
• Add score distributions to compute_perf
• Add reporting step to compute_perf
• Improved CI
• Use https checkout in develop.cfg

• Dropped support for Python 3.3
• Allow scripts to guess the number of columns in an input file
• CI fixes after Gitlab migration
• Simplified README
• Fix #14 (CMC test units)