• Breaking and significant changes:
  • Removed the old plotting scripts (compute_perf.py, etc.).
  • Added a new 2-column score format specific to bob.measure.
  • Implemented generic metrics and plotting scripts. Do bob measure --help to see the new scripts and refer to the documentation.
  • Matplotlib 2.2 and above is required now.
  • Some biometric-related-only functionality is moved to bob.bio.base.
  • Detailed changes:
  • !52 generic plotting script for bob measure: From #37, provide generic plotting scripts: * bob measure evaluate * bob measure hist * bob measure hter * ...
  • !55 Change option name criter to criterion
  • !56 Fix error in context name
  • !57 Extend bins number option
  • !58 Change variable name form criter to criterion
  • !59 Bug fix: incorrect input file reading
  • !60 Bugfix
  • !54 Refactors the score loading and scripts functionality: This merge request: * Move the biometric related functionality of bob.measure to bob.bio.base. * Add confidence interval calculations * Provide a score format for bob.measure with load functionalities * Provide a generic plotting script bob measure in bob.measure using bob.measure input file format: * bob measure metrics: to compute thresholds and evaluate performances * bob measure roc: to plot ROC * bob measure det: to plot DET * bob measure hist: to plot histograms * bob measure epc : to plot EPC * bob measure evaluate: applies all the above commands at once * bob measure gen: to generate fake scores for bob.measure Each command accepts one or several (dev,eval) score(s) file(s) for each system.
  • !61 Title and histograms subplots: Account for this bob.bio.base!146#note_28760.
  • Titles can be remove using an empty string, i.e. -t ' ' * Histograms support subplot display, see options --subplots * Add --legends-ncol option
  • !62 Improve legends in histograms
  • !48 recompute far values in roc_for_far: Fixes #27
  • !44 Compute roc using roc_for_far internally: Fixes #26
  • !43 Resolve "FAR and FRR thresholds are computed even when there is no data support": Closes #27 Also changes behavior of far_threshold and frr_threshold where the returned threshold guarantees the at most the requested far/frr value.
  • !63 Enable semilogx option in roc curves: * Fixes #40 * Remove 0 points on x-axis in semilogx plots. Matplotlib 2 was doing this automatically before but matplotlib 2.2 doesn't * Code clean-up
  • !64 Fix semilog plots for non numpy arrays: Fixes bob.bio.base#117
  • !65 Modification of criterion_option: Correct display of available criteria
  • !66 Add prefix aliasing: Prefix aliasing using AliasedGroup for bob.extension
  • !68 Change --eval option default and Various fixes: Change defaults, clean unused options for histograms. Fix bob.bio.base#112
  • !69 Explain that the thresholds might not satisfy the requested criteria: * Add a helper eer function for convenience.
  • !50 Generic loading input file: Add a generic input file loading function for bob measure and its corresponding test. From issue #39, comments #39#note_26366.
  • !71 Fix issues with matplotlib.hist.: For some reason mpl.hist hangs when n_bins is set to auto. This is related with https://github.com/numpy/numpy#8203 This MR set the default value to doane. Furthermore, allows you to pick one of the options here https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html#numpy.histogram This will fix one of the points here bob.pad.base!43
  • !72 Various improvements: Fix #41 and #42
  • !70 Improvements to new metrics codes
  • !73 Histogram legends: Fix #44
  • !74 Bins histograms: Option only depends on the number of data plotter per histograms, not the number of system. For example, pad hist requires 2 nbins and vuln hist requires 3 independently on the number of systems to be plotted. Fix #45.
  • !75 Fix issue with histo legends: Fix #47 Also change default for number of legend columns. Add comments and doc in the code for Hist.
  • !76 Histo fix: Fix typo and bob.pad.base!43#note_31884
  • !77 dd lines for dev histograms: See bob.pad.base!43#note_31903
  • !78 Metrics: All stuff related to #46.
  • !79 Add a command for multi protocol (N-fold cross validation) analysis
  • !80 Consider non 1 values negatives: This will make bob.measure scripts work with FOFRA scores
  • !81 Compute HTER using FMR and FNMR: As discussed in the meeting. Fixes #48
  • !83 Enable grid on all histograms
  • !84 Improve the constrained layout option
  • !82 Various fixes: * Change measure metrics * Document and change HTER * add decimal precision option for metric * Use acronyms instead of full names in figures * Remove filenames form figures and add log output instead
  • !86 Fix decimal number control for metrics: Fixes #52
  • !88 Fix tests: Should fix bob.nightlies#40
  • !67 Titles: Allow list of titles and remove (development) (evaluation) when default titles are modified
  • !45 Condapackage
  • !85 Fix broken commands cref bob.extension!86
  • !53 Change the way the scores arguments are passed to the compute() function: it now: Change the way the scores arguments are passed to the compute() function: it now does not rely on dev,eval pairs anymore and can take any number of different files (e.g. train)
  • !87 Update documentation and commands: FAR->FPR, FRR->FNR: Fix #54