Skip to content

MinDCF problem in negative sets with outlier scores

When we have an outlier score in negative set, the selected threshold in bob.measure.min_weighted_error_rate_threshold function is wrong. For example:

  from bob.measure import min_weighted_error_rate_threshold, farfrr
  cost = 0.99
  negatives = [-3, -2, -1, -0.5, 4]
  positives = [0.5, 3]
  th = min_weighted_error_rate_threshold(negatives, positives, cost, True)
  print("threshold: " + str(th))
  far, frr = farfrr(negatives, positives, th)
  mindcf = (cost*far + (1-cost)*frr)*100

  print ("minDCF : " + str(mindcf))

In this condition the output will be:

threshold: 0.0
minDCF : 19.8

minDCF can not be more than 1. In this condition a threshold higher than maximum score must be chosen. e.g., with threshold 5 minDCF will be 1.