Created mechanism that allows us to train only parts of the graph during the graph adaptation

Hey,

Sometimes when we want to apply some checkpoint in another dataset (or domain), we may want to choose which parts of the graph we want to re-train.

With this MR, our estimators (Logits.., Triplet, Siamese,) were enhanced with this keyword argument

        extra_checkpoint = {
            "checkpoint_path": <YOUR_CHECKPOINT>,
            "scopes": dict({"<SOURCE_SCOPE>/": "<TARGET_SCOPE>/"}),
            "trainable_variables": [<LIST OF VARIABLES OR SCOPES THAT YOU WANT TO RETRAIN>]
        }

The novelty here is the trainable_variables, where now we can set the parts of the graph we want to do back-propagation. If you set an empty list ( "trainable_variables": []) all variables will not be trainable. If this variable is not set at all, everything is trainable.

This is strongly dependent on how the architecture function is crafted. Look some example on how such functions should be crafted.

• Simple example: https://gitlab.idiap.ch/bob/bob.learn.tensorflow/blob/9a1cd4eda365f1470eab4ce9d8b49bb0278a0ae4/bob/learn/tensorflow/network/Dummy.py
• Complex one: https://gitlab.idiap.ch/bob/bob.learn.tensorflow/blob/9a1cd4eda365f1470eab4ce9d8b49bb0278a0ae4/bob/learn/tensorflow/network/InceptionResnetV2.py

Do you have some time to review this one @amohammadi ? Perhaps this can be useful for you.

Thanks

I've pylint some code too

I wouldn't use kwargs for this. You may want to use it to discard some extra arguments later.

changed this line in version 2 of the diff

this is good but I really don't like that you are using kwargs for something specific.

I'm doing like that just to have more flexibility and not to have a rigid API. I don't want to force this specific keyword argument in the estimators.

input, mode and scope are more orthodox, but everything else is superfluous. That's why I'm shipping this via kwargs

@tiago.pereira normally you would use kwargs to accept extra arguments that you don't need. This is useful when calling several different functions at the same time.

def f1(a=0, **kw):
  print(a)
def f2(b=0, **kw):
  print(b)

# call both using the same syntax
f1(a=a, b=b) # ignores b
f2(a=a, b=b) # ignores a

Different architectures take different parameters and I imagined the way to use them with logit trainer would be that all architectures accept kwargs but do not use what is inside kwargs.

Do we have anything blocking this to be merged? Thanks

added 1 commit

• 33d4e3b6 - Created mechanism that allows as to train only parts of the graph

Compare with previous version

added 25 commits

• 33d4e3b6...18980c2c - 24 commits from branch master
• 07b17ec9 - Merge branch 'master' into 'shutting-down-parts-of-the-network'

Compare with previous version

Hey @amohammadi, I've fixed the issues with kwargs. Now I think is cleaner.

Shall we move on with this?

Thanks

Is this comment still valid?

merged

mentioned in commit 69f0e7b0