Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in
bob.measure
bob.measure
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 2
    • Issues 2
    • List
    • Boards
    • Labels
    • Milestones
  • Merge Requests 1
    • Merge Requests 1
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • bob
  • bob.measurebob.measure
  • Issues
  • #3

Closed
Open
Opened Nov 13, 2015 by André Anjos@andre.anjos💬
  • Report abuse
  • New issue
Report abuse New issue

eer_threshold doesn't behave as expected

Created by: siebenkopf

Hi there,

I have found out that the eer_threshold function (as well as the min_hter_threshold function) does not perform as expected in case of highly unbalanced scores. Particularly, when I try to use a special low valued error indicator such as -10000, the returned threshold is garbage. Consider the following example:

positives = [1.]*100 + [-10000.]
negatives = [-1.]*100 + [-10000.]

threshold = bob.measure.eer_threshold(negatives, positives)
print threshold, bob.measure.farfrr(negatives, positives, threshold)
print 0, bob.measure.farfrr(negatives, positives, 0)

The outputs of the two print commands are:

-4999.5 (0.9900990099009901, 0.009900990099009901)
0 (0.0, 0.009900990099009901)

So, the second output, where I estimated a threshold 0 myself, is much more suitable than the first output, which was generated by eer_threshold (the min_hter_threshold behaves exactly the same).

I would suggest, we should rethink our implementation of the thresholds. I am pretty sure that there exist smarter ways (I had implemented some during my PhD...)

Cheers Manuel

Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
2
Labels
bug enhancement
Assign labels
  • View project labels
Reference: bob/bob.measure#3