• !99 Adjusted the histogram scale of the x axis to automatically adjust for varying scales across dev and eval

• !97 Add Area Under ROC Curve (AUC): Add the function called bob.measure.roc_auc_score. Add the AUC computation to bob measure metrics command. Fixes #2
• !98 Separate semilogx and TPR options in ROC plots: Also, ran black on some code files.

• !95 Improve threshold calculation in ROC, DET, and Precision and Recall plots: Fixes #58 and #60
• !96 Add an alpha option to ROC,DET,EPC figures: Also, enables you to change the default value in tablefmt option.
• Python 3.7 support

• !89 Add a base class for subplot-based plots
• !90 Fixing histogram test
• !91 Fix decimal option in metrics command: Fixes #52
• !43 Resolve "FAR and FRR thresholds are computed even when there is no data support"
• !92 Fix minimizing threshold: Closes #59
• !93 Click scripts fixes: Improvements and bug fixes to click scripts
• !94 Fix --figsize help message

• Breaking and significant changes:
• Removed the old plotting scripts (compute_perf.py, etc.).
• Added a new 2-column score format specific to bob.measure.
• Implemented generic metrics and plotting scripts. Do bob measure --help to see the new scripts and refer to the documentation.
• Matplotlib 2.2 and above is required now.
• Some biometric-related-only functionality is moved to bob.bio.base.
• Detailed changes:
• !52 generic plotting script for bob measure: From #37, provide generic plotting scripts: * bob measure evaluate * bob measure hist * bob measure hter * ...
• !55 Change option name criter to criterion
• !56 Fix error in context name
• !57 Extend bins number option
• !58 Change variable name form criter to criterion
• !59 Bug fix: incorrect input file reading
• !60 Bugfix
• !54 Refactors the score loading and scripts functionality: This merge request: * Move the biometric related functionality of bob.measure to bob.bio.base. * Add confidence interval calculations * Provide a score format for bob.measure with load functionalities * Provide a generic plotting script bob measure in bob.measure using bob.measure input file format: * bob measure metrics: to compute thresholds and evaluate performances * bob measure roc: to plot ROC * bob measure det: to plot DET * bob measure hist: to plot histograms * bob measure epc : to plot EPC * bob measure evaluate: applies all the above commands at once * bob measure gen: to generate fake scores for bob.measure Each command accepts one or several (dev,eval) score(s) file(s) for each system.
• !61 Title and histograms subplots: Account for this bob.bio.base!146#note_28760.

• Titles can be remove using an empty string, i.e. -t ' ' * Histograms support subplot display, see options --subplots * Add --legends-ncol option

• !62 Improve legends in histograms
• !48 recompute far values in roc_for_far: Fixes #27
• !44 Compute roc using roc_for_far internally: Fixes #26
• !43 Resolve "FAR and FRR thresholds are computed even when there is no data support": Closes #27 Also changes behavior of far_threshold and frr_threshold where the returned threshold guarantees the at most the requested far/frr value.
• !63 Enable semilogx option in roc curves: * Fixes #40 * Remove 0 points on x-axis in semilogx plots. Matplotlib 2 was doing this automatically before but matplotlib 2.2 doesn't * Code clean-up
• !64 Fix semilog plots for non numpy arrays: Fixes bob.bio.base#117
• !65 Modification of criterion_option: Correct display of available criteria
• !66 Add prefix aliasing: Prefix aliasing using AliasedGroup for bob.extension
• !68 Change --eval option default and Various fixes: Change defaults, clean unused options for histograms. Fix bob.bio.base#112
• !69 Explain that the thresholds might not satisfy the requested criteria: * Add a helper eer function for convenience.
• !50 Generic loading input file: Add a generic input file loading function for bob measure and its corresponding test. From issue #39, comments #39#note_26366.
• !71 Fix issues with matplotlib.hist.: For some reason mpl.hist hangs when n_bins is set to auto. This is related with https://github.com/numpy/numpy#8203 This MR set the default value to doane. Furthermore, allows you to pick one of the options here https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html#numpy.histogram This will fix one of the points here bob.pad.base!43
• !72 Various improvements: Fix #41 and #42
• !70 Improvements to new metrics codes
• !73 Histogram legends: Fix #44
• !74 Bins histograms: Option only depends on the number of data plotter per histograms, not the number of system. For example, pad hist requires 2 nbins and vuln hist requires 3 independently on the number of systems to be plotted. Fix #45.
• !75 Fix issue with histo legends: Fix #47 Also change default for number of legend columns. Add comments and doc in the code for Hist.
• !76 Histo fix: Fix typo and bob.pad.base!43#note_31884
• !77 dd lines for dev histograms: See bob.pad.base!43#note_31903
• !78 Metrics: All stuff related to #46.
• !79 Add a command for multi protocol (N-fold cross validation) analysis
• !80 Consider non 1 values negatives: This will make bob.measure scripts work with FOFRA scores
• !81 Compute HTER using FMR and FNMR: As discussed in the meeting. Fixes #48
• !83 Enable grid on all histograms
• !84 Improve the constrained layout option
• !82 Various fixes: * Change measure metrics * Document and change HTER * add decimal precision option for metric * Use acronyms instead of full names in figures * Remove filenames form figures and add log output instead
• !86 Fix decimal number control for metrics: Fixes #52
• !88 Fix tests: Should fix bob.nightlies#40
• !67 Titles: Allow list of titles and remove (development) (evaluation) when default titles are modified
• !45 Condapackage
• !85 Fix broken commands cref bob.extension!86
• !53 Change the way the scores arguments are passed to the compute() function: it now: Change the way the scores arguments are passed to the compute() function: it now does not rely on dev,eval pairs anymore and can take any number of different files (e.g. train)
• !87 Update documentation and commands: FAR->FPR, FRR->FNR: Fix #54

• Fixed issue with FAR and FRR values being exactly 1 or 0
• Corrected the threshold calculation in far_threshold() and frr_threshold(), which now guarantees to produce at most of given FAR or FRR values.
• Never return NaNs for far_threshold() and frr_threshold() functions
• Optimized implementation of ROC for a given FAR
• Added an option to capture the output in a log file
• Updated docs and tests accordingly

• Correctly handled NaN values in DIR plot
• Correctly handled of FAR and FRR thresholds
• Updated CMC documentation
• Switched to Numpy style documentation
• Fixed the dependency with bob.math (>=3.0.0)
• Fixed FAR and FRR threshold computation and return NaN when threshold cannot be computed

• !28 : Add an option to the epc function to also get the thresholds which were used during calculation

• Change the style of function comments
• Remove nan from scores and report FTA instead Report proper FMR, FNMR, FAR, FRR values
• Added min_weighted_error_rate_threshold in test for raised exceptions
• Implemented check on empty arrays for some functions -> raises excpetions now
• Used svg build badges
• Updated sphinx docs
• Load score more efficiently for negatives and positives
• Changed the ROC curve to implement what is required by 'The Handbook of Face Recognition'

This release brings some minor improvements to version 2.2.0 of this package:

• The use of a test-set on compute_perf is now optional
• Documentation concerning the scripts has been fixed to reflect changes on command-lines
• Implements bob.core verbosity controls on all scripts
• Fixes CMC plotting
• Re-organized score-loading
• Improved score-loading speeds

This release changes the command-line of most scripts in this package. Be attentive.

• Use docopt instead of argparse
• Make script interface more intuitive (remove non-optional options, use arguments)
• Fix all documentation warnings (nit-picky sphinx mode)
• Add score distributions to compute_perf
• Add reporting step to compute_perf
• Improved CI
• Use https checkout in develop.cfg

• Dropped support for Python 3.3
• Allow scripts to guess the number of columns in an input file
• CI fixes after Gitlab migration
• Simplified README
• Fix #14 (CMC test units)