I have a set of dev and eval scores and currently plotting an roc of them looks like this:

Here is the command I used: bin/bob measure roc -vvve scores-{dev,eval} --lines-at 1e-2 and here are the score files: scores-.zip

As you can see the dot in the eval set does not fall on the roc line.