Python-only version of bob.measure

This MR implements the following changes to this library, in an API backward-compatible way:

1. Minimize the number of Bob dependencies to a bare minimum. Use h5py to read data, RNGs and linear algebra from numpy. Remove requirements for bob.core, bob.io.base, bob.blitz, and most of bob.math (except for the PAVX algorithm)
2. Remove deprecated dependencies (docopt)
3. Use pytest for tests instead of nose
4. Re-implement the old C++ as a Python-only library (remove all C++ dependencies such as Boost)

added 1 commit

• 8eff2ada - [doc/guide] Zero warnings

Compare with previous version

@tiago.pereira: have a look, and if you can, test it.

changed the description

added 1 commit

• 36f4419a - [requirements] Pin bob.math to that pavx is available

Compare with previous version

Oh wow, that's a nice gift. Thank you very much @andre.anjos,

I'm happy to review it.

This closes #63 (closed)

About the PAVX I can myself port what we have on bob.math, but first, let me see if this implementation from scikit-learn is enough: https://scikit-learn.org/stable/modules/generated/sklearn.isotonic.IsotonicRegression.html

Please note I have not tested the implementations here, except for its own unit tests, which present 89% coverage, but still. It would be good to have a go at a more realistic scenario.

added 1 commit

• a95c7c72 - [conda] Remove build dependencies

Compare with previous version

resolved all threads

No feedbacks so far. @tiago.pereira: any thoughts?

Hey @andre.anjos, Yes, I'll review this one, no worries. I just need time.

What I'll do first is to branch the nightlies against this bob.measure and see how it goes

added 1 commit

• efc7f52e - [conda] Remove libboost and libblitz from requirements

Compare with previous version

I have looked at the Python implementation of the parts that I had implented, seems correct.

Thanks for this check.

There are things I'd like to bring in here:

1. I have a bunch of useful ML figures of merit on bob/bob.ip.binseg>, including plotting, and significance calculation I'd like to bring to this package and share.
2. What is today in _library, makes more sense in other sub-modules. For example, all plotting stuff can now be moved to the plot module.

Before doing these mods, I'd like that we validate this so I can start moving.

Ok, I'll do it tomorrow (my Bob day), if it's ok for you. I have a bunch of scores that I can use to compare. I'll test corner cases as well.

Please do so. Keep in mind it is going to be slow. It is a brute force implementation with unoptimised C-style code. If the speed is unbearable, we may have to consider Cythonizing some of it.

Yes, let's see. I have score files with more than 40M scores

The implementation looks perfect. Furthermore, I've tested with the scores I have, and everything matches (Error rates, decision thresholds, plots).

Performance-wise. My test was not suuuuuper scientific. Just for us have an idea.

My scores are in SCORES=/idiap/temp/tpereira/temp/

bob bio metrics

Script	C++	Python	#Scores
bob bio metrics -e $SCORES/mobio-male/scores-{dev,eval}	~7s	~16s	~212k
bob bio metrics $SCORES/arface-lfw/scores-dev	~11s	~40s	~736k
bob bio metrics $SCORES/morph/scores-dev	~4m	~38m	~44M

bob bio metrics evaluate (it generates all the plots, plus some metrics)

Script	C++	Python	#Scores
bob bio evaluate -e $SCORES/mobio-male/scores-{dev,eval}	~15s	~16s	~212k
bob bio evaluate $SCORES/arface-lfw/scores-dev	~20s	~40s	~736k
bob bio evaluate $SCORES/morph/scores-dev	~13m	-	~44M

I'll try to cythonize "_library.py" (never played with cython)

added 1 commit

• 84ac3013 - [cython]

Compare with previous version

added 2 commits

• b14d44ec - Fixed logger
• 75aab424 - Extended the artifact list

Compare with previous version