The low-level database File object is not kept in BioFile

The whole point of refactoring was to use the load method from the database but at some point we lose the object and cannot call its load method anymore. This was unfortunately a mistake by me.

Started branch 40-the-low-level-database-file-object-is-not-kept-in-biofile

I'm already there working. I can do it.

So the base signature from BioFile should be something like this:

    def __init__(self, client_id, path, file_base=None, file_id=None):
        """**Constructor Documentation**

        Initialize the File object with the minimum required data.

        Parameters:
        file_base: low level implementation of the file
        client_id : various type
          The id of the client this file belongs to.
          Its type depends on your implementation.
          If you use an SQL database, this should be an SQL type like Integer or String.
        For path and file_id, please refer to :py:class:`bob.db.base.File` constructor
        """

Reassigned to @tiago.pereira

This basically means, no release today :-(

I changed the constructor. I think it is better to make file_base=None, just in case of direct interaction with the HighLevel database.

class BioFile(bob.db.base.File):
    """A simple base class that defines basic properties of File object for the use in verification experiments"""

    def __init__(self, client_id, path, file_base=None, file_id=None):
        """**Constructor Documentation**

        Initialize the File object with the minimum required data.

        Parameters:
        file_base: low level implementation of the file
        client_id : various type
          The id of the client this file belongs to.
          Its type depends on your implementation.
          If you use an SQL database, this should be an SQL type like Integer or String.
        For path and file_id, please refer to :py:class:`bob.db.base.File` constructor
        """

@andre.anjos I was discussing with @amohammadi and we think that this issue is not necessary. The solution in place is already good.

Think that BioFile, in bob.bio.base, is like the Beat views. Beat needs a special data structure to work, so as bob.bio, and that's we have done for bob.bio.face, `bob.bio.spear, etc. The framework knows what it wants.

If your database has a weird way to open files and this crazy way is implemented in your database package, you can do the way that I suggested to Marc in (https://gitlab.idiap.ch/bob/bob.bio.spear/blob/add_nistsre12_db_/bob/bio/spear/database/nist_sre12.py).

I think we should close this issue and move on.

Very good explanation @tiago.pereira.

@andre.anjos The point is that databases are free to load their data however they want, but we want it in specific format in the verification framework. So here, you need to write a thin wrapper that converts your loaded data from the low-level format to a format that is required by the high level Preprocessor (no to mention that @mguenther insisted of having a way to have no wrapper at all and just read the data from the database by the preprocessor). This wrapper has to implemented for ALL databases in their high-level implementations.

We believe this wrapper is BioFile which by default implements a default wrapper that works with majority of databases that we have (e.g. io.base.load in FaceBioFile works with preprocessors in bio.face and all our face databases (so far)).

If that is not enough for you, your database should not return vanilla BioFile but rather a custom BioFile which has a custom load method and also can have a custom __init__ method too (to keep the low-level file if needed). A very good example is implemented already in nist_sre12 in bio.spear. See default wrapper implementation here: https://gitlab.idiap.ch/bob/bob.bio.spear/blob/add_nistsre12_db_/bob/bio/spear/database/database.py#L24 and a custom wrapper here: https://gitlab.idiap.ch/bob/bob.bio.spear/blob/add_nistsre12_db_/bob/bio/spear/database/nist_sre12.py#L28

Status changed to closed

Also, the problem before was that this wrapper (very similar to beat views) was integrated in the Preprocessor. So if you had a custom database, you had to write a different Preprocessor for it. This is fixed now by calling BioFileObject.load by default while this does not mean that this BioFileObject.load should be calling LowLevelFileObject.load by default.

Status changed to reopened

Hey guys,

I was thinking about this problem this morning and I decided to reopen the issue. I decided to draw a sequential diagram, so everybody can rely in this drawing instead in the one painted in our minds.

This sequential diagram is the ideal one. I think we should do something like that.

What we are doing today, is to eliminate the load function from bob.db.MyDatabase.File. This is ok for most of databases, since both, the bob.db.MyDatabase.File and bob.bio.xxx.MyDatabaseBioFile have the same inheritance in bob.db.base.File, but for some corner cases, this is not an elegant solution.

For instance a ideal solution (or at least close to the ideal) would be something like this (https://gitlab.idiap.ch/bob/bob.bio.spear/blob/add_nistsre12_db_/bob/bio/spear/database/nist_sre12.py#L28).

In short, we should use the load functions from bob.db.MyDatabase by default. If necessary, it is ok to do some specializations in the upper layers of the stack (something that we are already doing).

I don't agree. Only the wrapper layer knows what the database is loading and what does the preprocessor expects. We have talked about this! We never had an API for our bob.db.MyDatabase before we wanted databases to just use a few things from bob.db.base nothing is mandated there. We should have databases right now that don't have a load method.

I think it is a bit too late now for another iteration on this. I propose we release the package as is.

In any case, I think we should discuss that what is the low-level is supposed to do then we can discuss how to handle it in the high-level.

mentioned in merge request !42 (merged)

let's close this one.

Maybe in a refactoring 2018

Status changed to closed

mentioned in issue #39 (closed)

Mentioned in merge request bob.bio.face!18 (closed)