Don't jump to global settings when releasing middle-click

It shouldn't be doing this but somehow it is