Updated instructions

CLoses #4 (closed)

README.rst

@@ -80,9 +80,9 @@ Execute the code below to trigger the experiments::
   $ bob bio pipeline simple arface-all iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/arface/
   $ bob bio pipeline simple arface-all attentionnet -g dev -g eval -o ./results/arface/
   $ bob bio pipeline simple arface-all facenet-sanderberg -g dev -g eval -o ./results/arface/
-
- To generate the plot, do::
-
+  
+ To generate the plots from Figure 3, do::
+ 
   $ bob bio face plots arface -e ./results/arface/iresnet100/scores-{dev,eval}.csv \
     ./results/arface/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
     ./results/arface/attentionnet/scores-{dev,eval}.csv \
@@ -98,9 +98,9 @@ Execute the code below to trigger the experiments::
   $ bob bio pipeline simple mobio-all iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/mobio/
   $ bob bio pipeline simple mobio-all attentionnet -g dev -g eval -o ./results/mobio/
   $ bob bio pipeline simple mobio-all facenet-sanderberg -g dev -g eval -o ./results/mobio/
-
- To generate the plot, do::
-
+  
+ To generate the plots from Figure 7, do::
+ 
   $ bob bio face plots mobio-gender -e ./results/mobio/iresnet100/scores-{dev,eval}.csv \
     ./results/mobio/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
     ./results/mobio/attentionnet/scores-{dev,eval}.csv \
@@ -116,26 +116,33 @@ Execute the code below to trigger the experiments::
   $ bob bio pipeline simple multipie iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/multipie/
   $ bob bio pipeline simple multipie attentionnet -g dev -g eval -o ./results/multipie/
   $ bob bio pipeline simple multipie facenet-sanderberg -g dev -g eval -o ./results/multipie/
-
- To generate the plots, do::
-
+  
+ To generate the plots from Figure 4, do::
+ 
   $ bob bio face plots multipie-pose -e ./results/multipie/iresnet100/scores-{dev,eval}.csv \
     ./results/multipie/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
     ./results/multipie/attentionnet/scores-{dev,eval}.csv \
     ./results/multipie/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg
   $ bob bio face plots multipie-expression -e ./results/multipie/iresnet100/scores-{dev,eval}.csv \
     ./results/multipie/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
-    ./results/multipie/attentionnet/scores-{dev,eval}.csv \
+    ./results/multipie/attentionnet/scores-{dev,eval}.csv \ 
+    ./results/multipie/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg    
+    
+To generate the plot from Figure 9 (a), please, set the option `--optimal-threshold` on the `bob bio face plots multipie-pose` CLI command like the command below::
+
+  $ bob bio face plots multipie-pose --optimal-threshold -e ./results/multipie/iresnet100/scores-{dev,eval}.csv \
+    ./results/multipie/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
+    ./results/multipie/attentionnet/scores-{dev,eval}.csv \ 
     ./results/multipie/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg
+    
 
-We also have added two flags to have a protocol-dependent threshold, or to select the threshold on the eval set.
-Make sure that you **do not use these flags in production**::
+To generate the plot from Figure 9 (b), please, set the option `--threshold-eval` on the `bob bio face plots multipie-pose` CLI command like the command below::
 
-  $ bob bio face plots multipie-pose -e ./results/multipie/iresnet100/scores-{dev,eval}.csv \
+  $ bob bio face plots multipie-pose --threshold-eval -e ./results/multipie/iresnet100/scores-{dev,eval}.csv \
     ./results/multipie/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
-    ./results/multipie/attentionnet/scores-{dev,eval}.csv \
-    ./results/multipie/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg \
-    -o multipie-separate-thresholds.pdf --optimal-threshold
+    ./results/multipie/attentionnet/scores-{dev,eval}.csv \ 
+    ./results/multipie/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg
+
 
   $ bob bio face plots multipie-pose -e ./results/multipie/iresnet100/scores-{dev,eval}.csv \
     ./results/multipie/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
@@ -152,9 +159,9 @@ Execute the code below to trigger the experiments::
   $ bob bio pipeline simple scface iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/scface/
   $ bob bio pipeline simple scface attentionnet -g dev -g eval -o ./results/scface/
   $ bob bio pipeline simple scface facenet-sanderberg -g dev -g eval -o ./results/scface/
-
- To generate the plot, do::
-
+  
+ To generate the plot from Figure 5, do::
+ 
   $ bob bio face plots scface-distance -e ./results/scface/iresnet100/scores-{dev,eval}.csv \
     ./results/scface/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
     ./results/scface/attentionnet/scores-{dev,eval}.csv \
@@ -170,18 +177,62 @@ Execute the code below to trigger the experiments::
   $ bob bio pipeline simple ijbc-test4-g1 iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/ijbc-test4-g1/
   $ bob bio pipeline simple ijbc-test4-g1 attentionnet -g dev -g eval -o ./results/ijbc-test4-g1/
   $ bob bio pipeline simple ijbc-test4-g1 facenet-sanderberg -g dev -g eval -o ./results/ijbc-test4-g1/
-
- To generate the plot, do::
-
+  
+ To generate the plot from Figure 8, do::
+ 
   $ bob bio face plots ijbc ./results/ijbc-test4-g1/iresnet100/scores-{dev,eval}.csv \
     ./results/ijbc-test4-g1/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
     ./results/ijbc-test4-g1/attentionnet/scores-{dev,eval}.csv \
     ./results/ijbc-test4-g1/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg
 
 
+   
+GBU experiments
+===============
+
+
+Execute the code below to trigger the experiments for the **good** protocol::
+
+  $ bob bio pipeline simple gbu-good iresnet100 -g dev -g eval -o ./results/gbu-good/
+  $ bob bio pipeline simple gbu-good iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/gbu-good/
+  $ bob bio pipeline simple gbu-good attentionnet -g dev -g eval -o ./results/gbu-good/
+  $ bob bio pipeline simple gbu-good facenet-sanderberg -g dev -g eval -o ./results/gbu-good/
+
+Execute the code below to trigger the experiments for the **bad** protocol::
+
+  $ bob bio pipeline simple gbu-bad iresnet100 -g dev -g eval -o ./results/gbu-bad/
+  $ bob bio pipeline simple gbu-bad iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/gbu-bad/
+  $ bob bio pipeline simple gbu-bad attentionnet -g dev -g eval -o ./results/gbu-bad/
+  $ bob bio pipeline simple gbu-bad facenet-sanderberg -g dev -g eval -o ./results/gbu-bad/
+
+
+Execute the code below to trigger the experiments for the **ugly** protocol::
+
+  $ bob bio pipeline simple gbu-ugly iresnet100 -g dev -g eval -o ./results/gbu-ugly/
+  $ bob bio pipeline simple gbu-ugly iresnet50-msceleb-idiap-20210623 -g dev -g eval -o ./results/gbu-ugly/
+  $ bob bio pipeline simple gbu-ugly attentionnet -g dev -g eval -o ./results/gbu-ugly/
+  $ bob bio pipeline simple gbu-ugly facenet-sanderberg -g dev -g eval -o ./results/gbu-ugly/
+
 
+  
+ To generate the plots for each sub-protocol from Figure 6, do::
+ 
+  $ bob bio face plots gbu ./results/gbu-good/iresnet100/scores-{dev,eval}.csv \
+    ./results/gbu-good/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
+    ./results/gbu-good/attentionnet/scores-{dev,eval}.csv \ 
+    ./results/gbu-good/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg -o GBU-good.pdf
 
+  $ bob bio face plots gbu ./results/gbu-bad/iresnet100/scores-{dev,eval}.csv \
+    ./results/gbu-bad/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
+    ./results/gbu-bad/attentionnet/scores-{dev,eval}.csv \ 
+    ./results/gbu-bad/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg -o GBU-bad.pdf
 
+  $ bob bio face plots gbu ./results/gbu-ugly/iresnet100/scores-{dev,eval}.csv \
+    ./results/gbu-ugly/iresnet50-msceleb-idiap-20210623/scores-{dev,eval}.csv \
+    ./results/gbu-ugly/attentionnet/scores-{dev,eval}.csv \ 
+    ./results/gbu-ugly/facenet-sanderberg/scores-{dev,eval}.csv --titles ArcFace-100,Idiap-Resnet50,Zoo-AttentionNet,Facenet-Sandberg -o GBU-ugly.pdf
+   
+   
 
 Contact
 -------