Tags give the ability to mark specific points in history as being important
- Breaking and significant changes:
- Removed the old plotting scripts (
- Added a new 2-column score format specific to bob.measure.
- Implemented generic metrics and plotting scripts. Do
bob measure --helpto see the new scripts and refer to the documentation.
- Matplotlib 2.2 and above is required now.
- Some biometric-related-only functionality is moved to bob.bio.base.
- Detailed changes:
- !52 generic plotting script for bob measure: From #37, provide generic plotting scripts: * bob measure evaluate * bob measure hist * bob measure hter * ...
- !55 Change option name criter to criterion
- !56 Fix error in context name
- !57 Extend bins number option
- !58 Change variable name form criter to criterion
- !59 Bug fix: incorrect input file reading
- !60 Bugfix
!54 Refactors the score loading and scripts functionality:
This merge request: * Move the biometric related functionality of
bob.measure to bob.bio.base. * Add confidence interval calculations *
Provide a score format for bob.measure with load functionalities *
Provide a generic plotting script bob measure in bob.measure using
bob.measure input file format: *
bob measure metrics: to compute thresholds and evaluate performances *
bob measure roc: to plot ROC *
bob measure det: to plot DET *
bob measure hist: to plot histograms *
bob measure epc: to plot EPC *
bob measure evaluate: applies all the above commands at once *
bob measure gen: to generate fake scores for
bob.measureEach command accepts one or several (dev,eval) score(s) file(s) for each system.
- !61 Title and histograms subplots: Account for this bob.bio.base!146#note_28760.
- Titles can be remove using an empty string, i.e.
-t ' '* Histograms support subplot display, see options
- !62 Improve legends in histograms
- !48 recompute far values in roc_for_far: Fixes #27
- !44 Compute roc using roc_for_far internally: Fixes #26
- !43 Resolve "FAR and FRR thresholds are computed even when there is no data support": Closes #27 Also changes behavior of far_threshold and frr_threshold where the returned threshold guarantees the at most the requested far/frr value.
- !63 Enable semilogx option in roc curves: * Fixes #40 * Remove 0 points on x-axis in semilogx plots. Matplotlib 2 was doing this automatically before but matplotlib 2.2 doesn't * Code clean-up
- !64 Fix semilog plots for non numpy arrays: Fixes bob.bio.base#117
- !65 Modification of criterion_option: Correct display of available criteria
- !66 Add prefix aliasing: Prefix aliasing using AliasedGroup for bob.extension
- !68 Change --eval option default and Various fixes: Change defaults, clean unused options for histograms. Fix bob.bio.base#112
- !69 Explain that the thresholds might not satisfy the requested criteria: * Add a helper eer function for convenience.
- !50 Generic loading input file: Add a generic input file loading function for bob measure and its corresponding test. From issue #39, comments #39#note_26366.
!71 Fix issues with matplotlib.hist.: For some reason
n_binsis set to
auto. This is related with https://github.com/numpy/numpy#8203 This MR set the default value to
doane. Furthermore, allows you to pick one of the options here https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html#numpy.histogram This will fix one of the points here bob.pad.base!43
- !72 Various improvements: Fix #41 and #42
- !70 Improvements to new metrics codes
- !73 Histogram legends: Fix #44
- !74 Bins histograms: Option only depends on the number of data plotter per histograms, not the number of system. For example, pad hist requires 2 nbins and vuln hist requires 3 independently on the number of systems to be plotted. Fix #45.
- !75 Fix issue with histo legends: Fix #47 Also change default for number of legend columns. Add comments and doc in the code for Hist.
- !76 Histo fix: Fix typo and bob.pad.base!43#note_31884
- !77 dd lines for dev histograms: See bob.pad.base!43#note_31903
- !78 Metrics: All stuff related to #46.
- !79 Add a command for multi protocol (N-fold cross validation) analysis
- !80 Consider non 1 values negatives: This will make bob.measure scripts work with FOFRA scores
- !81 Compute HTER using FMR and FNMR: As discussed in the meeting. Fixes #48
- !83 Enable grid on all histograms
- !84 Improve the constrained layout option
- !82 Various fixes: * Change measure metrics * Document and change HTER * add decimal precision option for metric * Use acronyms instead of full names in figures * Remove filenames form figures and add log output instead
- !86 Fix decimal number control for metrics: Fixes #52
- !88 Fix tests: Should fix bob.nightlies#40
!67 Titles: Allow list of titles and remove
(evaluation)when default titles are modified
- !45 Condapackage
- !85 Fix broken commands cref bob.extension!86
- !53 Change the way the scores arguments are passed to the compute() function: it now: Change the way the scores arguments are passed to the compute() function: it now does not rely on dev,eval pairs anymore and can take any number of different files (e.g. train)
- !87 Update documentation and commands: FAR->FPR, FRR->FNR: Fix #54
- Fixed issue with FAR and FRR values being exactly 1 or 0
- Corrected the threshold calculation in far_threshold() and frr_threshold(), which now guarantees to produce at most of given FAR or FRR values.
- Never return NaNs for far_threshold() and frr_threshold() functions
- Optimized implementation of ROC for a given FAR
- Added an option to capture the output in a log file
- Updated docs and tests accordingly
- Correctly handled NaN values in DIR plot
- Correctly handled of FAR and FRR thresholds
- Updated CMC documentation
- Switched to Numpy style documentation
- Fixed the dependency with bob.math (>=3.0.0)
- Fixed FAR and FRR threshold computation and return NaN when threshold cannot be computed
- !28 : Add an option to the epc function to also get the thresholds which were used during calculation
- Change the style of function comments
- Remove nan from scores and report FTA instead Report proper FMR, FNMR, FAR, FRR values
- Added min_weighted_error_rate_threshold in test for raised exceptions
- Implemented check on empty arrays for some functions -> raises excpetions now
- Used svg build badges
- Updated sphinx docs
- Load score more efficiently for negatives and positives
- Changed the ROC curve to implement what is required by 'The Handbook of Face Recognition'
This release brings some minor improvements to version 2.2.0 of this package:
- The use of a test-set on
compute_perfis now optional
- Documentation concerning the scripts has been fixed to reflect changes on command-lines
- Implements bob.core verbosity controls on all scripts
- Fixes CMC plotting
- Re-organized score-loading
- Improved score-loading speeds
- The use of a test-set on
This release changes the command-line of most scripts in this package. Be attentive.
- Use docopt instead of argparse
- Make script interface more intuitive (remove non-optional options, use arguments)
- Fix all documentation warnings (nit-picky sphinx mode)
- Add score distributions to
- Add reporting step to
- Improved CI
- Use https checkout in
- Dropped support for Python 3.3
- Allow scripts to guess the number of columns in an input file
- CI fixes after Gitlab migration
- Simplified README
- Fix #14 (CMC test units)