Load score more efficiently for negatives and positives

Fixes #19 (closed)

@mguenther I think this is better than what you are trying to do. What do you think?