Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
bob.measure
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
2
Issues
2
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
bob
bob.measure
Commits
190eb6de
Commit
190eb6de
authored
May 03, 2018
by
Amir MOHAMMADI
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
lint
parent
5df44007
Changes
3
Expand all
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
144 additions
and
80 deletions
+144
-80
bob/measure/script/commands.py
bob/measure/script/commands.py
+23
-17
bob/measure/script/common_options.py
bob/measure/script/common_options.py
+58
-21
bob/measure/script/figure.py
bob/measure/script/figure.py
+63
-42
No files found.
bob/measure/script/commands.py
View file @
190eb6de
...
@@ -8,6 +8,7 @@ from . import common_options
...
@@ -8,6 +8,7 @@ from . import common_options
from
bob.extension.scripts.click_helper
import
(
verbosity_option
,
from
bob.extension.scripts.click_helper
import
(
verbosity_option
,
open_file_mode_option
)
open_file_mode_option
)
@
click
.
command
()
@
click
.
command
()
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
eval_option
()
@
common_options
.
eval_option
()
...
@@ -24,9 +25,9 @@ def metrics(ctx, scores, evaluation, **kwargs):
...
@@ -24,9 +25,9 @@ def metrics(ctx, scores, evaluation, **kwargs):
"""Prints a table that contains FtA, FAR, FRR, FMR, FMNR, HTER for a given
"""Prints a table that contains FtA, FAR, FRR, FMR, FMNR, HTER for a given
threshold criterion (eer or min-hter).
threshold criterion (eer or min-hter).
You need to provide one or more development score file(s) for each
experiment.
You need to provide one or more development score file(s) for each
You can also provide evaluation files along with dev files. If only dev scores
experiment. You can also provide evaluation files along with dev files. If
are provided, you must use flag `--no-evaluation`.
only dev scores
are provided, you must use flag `--no-evaluation`.
Resulting table format can be changed using the `--tablefmt`.
Resulting table format can be changed using the `--tablefmt`.
...
@@ -40,6 +41,7 @@ def metrics(ctx, scores, evaluation, **kwargs):
...
@@ -40,6 +41,7 @@ def metrics(ctx, scores, evaluation, **kwargs):
process
=
figure
.
Metrics
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
=
figure
.
Metrics
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
.
run
()
process
.
run
()
@
click
.
command
()
@
click
.
command
()
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
title_option
()
@
common_options
.
title_option
()
...
@@ -66,9 +68,9 @@ def roc(ctx, scores, evaluation, **kwargs):
...
@@ -66,9 +68,9 @@ def roc(ctx, scores, evaluation, **kwargs):
false non match rate on the vertical axis. The values for the axis will be
false non match rate on the vertical axis. The values for the axis will be
computed using :py:func:`bob.measure.roc`.
computed using :py:func:`bob.measure.roc`.
You need to provide one or more development score file(s) for each
experiment.
You need to provide one or more development score file(s) for each
You can also provide evaluation files along with dev files. If only dev scores
experiment. You can also provide evaluation files along with dev files. If
are provided, you must use flag `--no-evaluation`.
only dev scores
are provided, you must use flag `--no-evaluation`.
Examples:
Examples:
$ bob measure roc dev-scores
$ bob measure roc dev-scores
...
@@ -81,6 +83,7 @@ def roc(ctx, scores, evaluation, **kwargs):
...
@@ -81,6 +83,7 @@ def roc(ctx, scores, evaluation, **kwargs):
process
=
figure
.
Roc
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
=
figure
.
Roc
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
.
run
()
process
.
run
()
@
click
.
command
()
@
click
.
command
()
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
output_plot_file_option
(
default_out
=
'det.pdf'
)
@
common_options
.
output_plot_file_option
(
default_out
=
'det.pdf'
)
...
@@ -106,9 +109,9 @@ def det(ctx, scores, evaluation, **kwargs):
...
@@ -106,9 +109,9 @@ def det(ctx, scores, evaluation, **kwargs):
modified ROC curve which plots error rates on both axes
modified ROC curve which plots error rates on both axes
(false positives on the x-axis and false negatives on the y-axis)
(false positives on the x-axis and false negatives on the y-axis)
You need to provide one or more development score file(s) for each
experiment.
You need to provide one or more development score file(s) for each
You can also provide evaluation files along with dev files. If only dev scores
experiment. You can also provide evaluation files along with dev files. If
are provided, you must use flag `--no-evaluation`.
only dev scores
are provided, you must use flag `--no-evaluation`.
Examples:
Examples:
$ bob measure det dev-scores
$ bob measure det dev-scores
...
@@ -121,6 +124,7 @@ def det(ctx, scores, evaluation, **kwargs):
...
@@ -121,6 +124,7 @@ def det(ctx, scores, evaluation, **kwargs):
process
=
figure
.
Det
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
=
figure
.
Det
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
.
run
()
process
.
run
()
@
click
.
command
()
@
click
.
command
()
@
common_options
.
scores_argument
(
min_arg
=
1
,
force_eval
=
True
,
nargs
=-
1
)
@
common_options
.
scores_argument
(
min_arg
=
1
,
force_eval
=
True
,
nargs
=-
1
)
@
common_options
.
output_plot_file_option
(
default_out
=
'epc.pdf'
)
@
common_options
.
output_plot_file_option
(
default_out
=
'epc.pdf'
)
...
@@ -152,6 +156,7 @@ def epc(ctx, scores, **kwargs):
...
@@ -152,6 +156,7 @@ def epc(ctx, scores, **kwargs):
process
=
figure
.
Epc
(
ctx
,
scores
,
True
,
load
.
split
)
process
=
figure
.
Epc
(
ctx
,
scores
,
True
,
load
.
split
)
process
.
run
()
process
.
run
()
@
click
.
command
()
@
click
.
command
()
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
output_plot_file_option
(
default_out
=
'hist.pdf'
)
@
common_options
.
output_plot_file_option
(
default_out
=
'hist.pdf'
)
...
@@ -174,9 +179,9 @@ def hist(ctx, scores, evaluation, **kwargs):
...
@@ -174,9 +179,9 @@ def hist(ctx, scores, evaluation, **kwargs):
""" Plots histograms of positive and negatives along with threshold
""" Plots histograms of positive and negatives along with threshold
criterion.
criterion.
You need to provide one or more development score file(s) for each
experiment.
You need to provide one or more development score file(s) for each
You can also provide evaluation files along with dev files. If only dev scores
experiment. You can also provide evaluation files along with dev files. If
are provided, you must use flag `--no-evaluation`.
only dev scores
are provided, you must use flag `--no-evaluation`.
By default, when eval-scores are given, only eval-scores histograms are
By default, when eval-scores are given, only eval-scores histograms are
displayed with threshold line
displayed with threshold line
...
@@ -194,6 +199,7 @@ def hist(ctx, scores, evaluation, **kwargs):
...
@@ -194,6 +199,7 @@ def hist(ctx, scores, evaluation, **kwargs):
process
=
figure
.
Hist
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
=
figure
.
Hist
(
ctx
,
scores
,
evaluation
,
load
.
split
)
process
.
run
()
process
.
run
()
@
click
.
command
()
@
click
.
command
()
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
scores_argument
(
nargs
=-
1
)
@
common_options
.
legends_option
()
@
common_options
.
legends_option
()
...
@@ -217,8 +223,8 @@ def evaluate(ctx, scores, evaluation, **kwargs):
...
@@ -217,8 +223,8 @@ def evaluate(ctx, scores, evaluation, **kwargs):
\
b
\
b
1. Computes the threshold using either EER or min. HTER criteria on
1. Computes the threshold using either EER or min. HTER criteria on
development set scores
development set scores
2. Applies the above threshold on evaluation set scores to compute the
HTER, if a
2. Applies the above threshold on evaluation set scores to compute the
eval-score set is provided
HTER, if a
eval-score set is provided
3. Reports error rates on the console
3. Reports error rates on the console
4. Plots ROC, EPC, DET curves and score distributions to a multi-page PDF
4. Plots ROC, EPC, DET curves and score distributions to a multi-page PDF
file
file
...
@@ -261,12 +267,12 @@ def evaluate(ctx, scores, evaluation, **kwargs):
...
@@ -261,12 +267,12 @@ def evaluate(ctx, scores, evaluation, **kwargs):
click
.
echo
(
"Starting evaluate with dev scores only..."
)
click
.
echo
(
"Starting evaluate with dev scores only..."
)
click
.
echo
(
"Computing ROC..."
)
click
.
echo
(
"Computing ROC..."
)
# set axes limits for ROC
# set axes limits for ROC
ctx
.
forward
(
roc
)
# use class defaults plot settings
ctx
.
forward
(
roc
)
# use class defaults plot settings
click
.
echo
(
"Computing DET..."
)
click
.
echo
(
"Computing DET..."
)
ctx
.
forward
(
det
)
# use class defaults plot settings
ctx
.
forward
(
det
)
# use class defaults plot settings
if
evaluation
:
if
evaluation
:
click
.
echo
(
"Computing EPC..."
)
click
.
echo
(
"Computing EPC..."
)
ctx
.
forward
(
epc
)
# use class defaults plot settings
ctx
.
forward
(
epc
)
# use class defaults plot settings
# the last one closes the file
# the last one closes the file
ctx
.
meta
[
'closef'
]
=
True
ctx
.
meta
[
'closef'
]
=
True
click
.
echo
(
"Computing score histograms..."
)
click
.
echo
(
"Computing score histograms..."
)
...
...
bob/measure/script/common_options.py
View file @
190eb6de
This diff is collapsed.
Click to expand it.
bob/measure/script/figure.py
View file @
190eb6de
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment