[ci] Simplify

4 jobs for python-only in 1 minute and 20 seconds (queued for 5 seconds)