High-level db interfaces here are not compatible with bob.bio.video

When having a look into the implementation of bob.bio.video, you can see two details:

The frame_selector is a parameter to the load function: https://gitlab.idiap.ch/bob/bob.bio.video/blob/0ba8960898586974ad5da0b1d64d09406a672b12/bob/bio/video/database/database.py#L18
The frame_selector is provided in the default implementation of read_original_data: https://gitlab.idiap.ch/bob/bob.bio.video/blob/0ba8960898586974ad5da0b1d64d09406a672b12/bob/bio/video/preprocessor/Wrapper.py#L62 and the FrameSelector can be specified in the constructor (including any type of selection_style): https://gitlab.idiap.ch/bob/bob.bio.video/blob/0ba8960898586974ad5da0b1d64d09406a672b12/bob/bio/video/preprocessor/Wrapper.py#L32

@amohammadi While you are correct that the result cannot be processed directly by the bob.bio.video wrapper classes, you can still use them indirectly by providing a custom read_original_data function. Nevertheless, I think we should have the functions to be compatible with bob.bio.video, i.e., by providing the frame_selector as a parameter to the load function.

Yes @mguenther but the way it is written as of now, you cannot load 20 frames instead of all frames for example.

You can, but it requires a custom implementation of read_original_data that does not use ReplayPad.load, which defeats the purpose of the implementation of ReplayPad.load. Hence, updating the function should be done.

@amohammadi, the issue is open for HLDI (High-level db interfaces). In my opinion, returning 20 frames is not the task of HLDI. HLDI must return all available frames, preprocessor can do sub-selection if needed.

Also, @amohammadi, not sure about this sentence: "you cannot use bob.bio.video wrapper classes". This is exactly the opposite. The load method returns Frame Container, which is directly acceptable by bob.bio.video wrappers. Have I misunderstood you?

I disagree oleg. The code should be written in a way that is flexible. Of course as Manuel pointed out you can hack it through to load for example 20 frames but that is not the purpose here. The goal here is to make these dbs compatible with bob.bio.video which will eventually enable you to fix #9 (closed) as well.

You can try running a pipeline with a configuration like this:

database = 'replay-attack'
from bob.pad.face.config.lbp_svm import preprocessor, algorithm
from bob.pad.face.extractor import LBPHistogram
import bob.bio.video
extractor = bob.bio.video.extractor.Wrapper(LBPHistogram())

And you will see what I mean.

assigned to @amohammadi

Current implementations of HLDIs work flawlessly in my use cases, also supporting the wrappers of bob.bio.video . @amohammadi, I am not aware of your specific requirements (frame sub-selection, etc.) for high-level interfaces, thus I assign this issue to you. Overall, I think it is a good idea to make the software flexible, matching requirements of different users. My only concern in the case of re-factoring: please, make sure that experimental results (see documentation) don't change. Also, modifying one of HLDIs might brake the Aggregated database, special attention is needed there.

Current implementations of HLDIs work flawlessly in my use cases

I disagree @onikisins! The way it is implemented now you need to implement two classes for every image-based preprocessor, extractor, and algorithm that you have. This is clear as you already have done so. For example:

There is a ImageQualityMeasure and a VideoQualityMeasure which just calls the other one and is completely unnecessary.
There is a LBPHistogram and a VideoLBPHistogram which is again the same thing.

and this list is going to grow when new image-based classes are implemented here. This is exactly what @andre.anjos was discussing with you about Don't Repeat Yourself or DRY. The video versions of these classes are not necessary since you can wrap them in bob.bio.video classes in configuration files to make them a video class.

Now, I am trying to explain here through this issue and issue #9 (closed) that the way bob.bio.video is meant to be used is like wrappers. If you still don't understand how this is meant to be done, I or another from the group can explain this to you. I have no intention of refactoring this package. I have already spent a considerable amount of my time reviewing this package, pointing out the issues, and explaining them. If you want to ignore them or do not think these are issues, please close them.

removed assignee

The issue is on HLDI, not preprocessors, extractors, ..., let's not pin them here. The HLDIs, as implemented now, are returning videos as FrameContainers. This is exactly the same as in bob.bio.video , just the implementation is different. @amohammadi, agree on that? Thus, all functionality of bob.bio.video is preserved. Don't see the problem here. In my opinion, this issue is just about the "beauty" of implementation, which is very subjective. If @amohammadi still disagree with me, I suggest we discuss it verbally, saving time for both of us. If no activity here I will close the issue later.

@onikisins please follow the example below: This is the configuration file:

import bob.bio.video
from bob.pad.face.config.replay_mobile import database

sub_directory = 'lbp_svm_video'

from bob.pad.face.preprocessor import ImageFaceCrop

preprocessor = bob.bio.video.preprocessor.Wrapper(ImageFaceCrop(64))

from bob.pad.face.config.lbp_svm import extractor, algorithm

And this is the command that I am running:

bin/spoof.py lbp_svm_video.py                                                                                                               
bob.pad.base@2017-10-18 10:07:49,973 -- ERROR: During the execution, an exception was raised: load() takes at most 3 arguments (4 given)        
Traceback (most recent call last):                                                                                                              
  File "bin/spoof.py", line 28, in <module>                                                                                                     
    sys.exit(bob.pad.base.script.spoof.main())                                                                                                  
  File "/remote/idiap.svm/user.active/amohammadi/git/tutorials/dap/src/bob.pad.base/bob/pad/base/script/spoof.py", line 299, in main            
    detect_spoofing(args, command_line_parameters)                                                                                              
  File "/remote/idiap.svm/user.active/amohammadi/git/tutorials/dap/src/bob.pad.base/bob/pad/base/script/spoof.py", line 273, in detect_spoofing 
    if not execute(args):                                                                                                                       
  File "/remote/idiap.svm/user.active/amohammadi/git/tutorials/dap/src/bob.pad.base/bob/pad/base/script/spoof.py", line 168, in execute         
    force=args.force)                                                                                                                           
  File "/remote/idiap.svm/user.active/amohammadi/git/tutorials/dap/src/bob.pad.base/bob/pad/base/tools/preprocessor.py", line 86, in preprocess 
    file_object, original_directory, original_extension)                                                                                        
  File "/remote/idiap.svm/user.active/amohammadi/conda/envs/bob3py27/lib/python2.7/site-packages/bob/bio/video/preprocessor/Wrapper.py", line 6 
2, in _read_video_data                                                                                                                          
    return biofile.load(directory, extension, frame_selector)                                                                                   
TypeError: load() takes at most 3 arguments (4 given)

What do you think is the problem here?

@amohammadi , nice example, thanks, that clarifies a lot! Posting this example in the beginning of the track would save lots of time for us. Your use cases are clear to me now. Yes, I am totally agree now, this is an issue worth to be addressed, will not close it. Thanks!

I'm glad it helped Olegs. I can explain other issue here as well if needed.

created branch 10-high-level-db-interfaces-here-are-not-compatible-with-bob-bio-video

mentioned in merge request !28 (merged)

assigned to @amohammadi

closed via merge request !28 (merged)

closed via commit 827cfd62

mentioned in commit 827cfd62

High-level db interfaces here are not compatible with bob.bio.video

Designs

Child items ...

Activity