Commit b4976267 authored by Theophile GENTILHOMME's avatar Theophile GENTILHOMME

Small changes in the user guide, add titles option to hist command

parent 14e852f2
Pipeline #18919 passed with stage
in 33 minutes and 50 seconds
...@@ -152,6 +152,7 @@ def epc(ctx, scores, **kwargs): ...@@ -152,6 +152,7 @@ def epc(ctx, scores, **kwargs):
@common_options.const_layout_option() @common_options.const_layout_option()
@common_options.show_dev_option() @common_options.show_dev_option()
@common_options.print_filenames_option() @common_options.print_filenames_option()
@common_options.titles_option()
@verbosity_option() @verbosity_option()
@click.pass_context @click.pass_context
def hist(ctx, scores, evaluation, **kwargs): def hist(ctx, scores, evaluation, **kwargs):
......
...@@ -491,6 +491,7 @@ on a set, after setting up |project|, just do: ...@@ -491,6 +491,7 @@ on a set, after setting up |project|, just do:
==== =================== ==== ===================
.. Development dev-1 .. Development dev-1
==== =================== ==== ===================
FtA 0.000%
FMR 6.263% (31/495) FMR 6.263% (31/495)
FNMR 6.208% (28/451) FNMR 6.208% (28/451)
FAR 5.924% FAR 5.924%
...@@ -498,7 +499,7 @@ on a set, after setting up |project|, just do: ...@@ -498,7 +499,7 @@ on a set, after setting up |project|, just do:
HTER 8.599% HTER 8.599%
==== =================== ==== ===================
The output will present the threshold together with the FMR, FMNR, FAR, FRR and The output will present the threshold together with the FtA, FMR, FMNR, FAR, FRR and
HTER on the given set, calculated using such a threshold. The relative counts of FAs HTER on the given set, calculated using such a threshold. The relative counts of FAs
and FRs are also displayed between parenthesis. and FRs are also displayed between parenthesis.
...@@ -519,6 +520,7 @@ To evaluate the performance of a new score file with a given threshold, use ...@@ -519,6 +520,7 @@ To evaluate the performance of a new score file with a given threshold, use
==== ==================== ==== ====================
.. Development eval-1 .. Development eval-1
==== ==================== ==== ====================
FtA 0.000%
FMR 5.010% (24/479) FMR 5.010% (24/479)
FNMR 6.977% (33/473) FNMR 6.977% (33/473)
FAR 4.770% FAR 4.770%
...@@ -536,6 +538,7 @@ on an evaluation set: ...@@ -536,6 +538,7 @@ on an evaluation set:
==== =================== =============== ==== =================== ===============
.. Development dev-1 Eval. eval-1 .. Development dev-1 Eval. eval-1
==== =================== =============== ==== =================== ===============
FtA 0.000% 0.000%
FMR 6.263% (31/495) 5.637% (27/479) FMR 6.263% (31/495) 5.637% (27/479)
FNMR 6.208% (28/451) 6.131% (29/473) FNMR 6.208% (28/451) 6.131% (29/473)
FAR 5.924% 5.366% FAR 5.924% 5.366%
...@@ -544,11 +547,8 @@ on an evaluation set: ...@@ -544,11 +547,8 @@ on an evaluation set:
==== =================== =============== ==== =================== ===============
.. note:: .. note::
Table format can be changed using ``--tablefmt`` option. Note that Table format can be changed using ``--tablefmt`` option, the default format
``latex`` being ``rst``. Please refer to ``bob measure metrics --help`` for more details.
format is used by default when the outputs are witten in a log file and
``rst`` when written in the terminal. See
``bob measure metrics --help`` for more details.
Plots Plots
...@@ -573,7 +573,7 @@ For example, to generate a DET curve from development and evaluation datasets: ...@@ -573,7 +573,7 @@ For example, to generate a DET curve from development and evaluation datasets:
.. code-block:: sh .. code-block:: sh
$bob measure det --split --output 'my_det.pdf' dev-1.txt eval-1.txt $bob measure det --output 'my_det.pdf' dev-1.txt eval-1.txt
dev-2.txt eval-2.txt dev-2.txt eval-2.txt
where `my_det.pdf` will contain DET plots for the two experiments. where `my_det.pdf` will contain DET plots for the two experiments.
...@@ -586,14 +586,14 @@ where `my_det.pdf` will contain DET plots for the two experiments. ...@@ -586,14 +586,14 @@ where `my_det.pdf` will contain DET plots for the two experiments.
Evaluate Evaluate
======== ========
A convenient command `evaluate` is provided to generate multiple metrics and A convenient command ``evaluate`` is provided to generate multiple metrics and
plots for a list of experiments. It generates two `metrics` outputs with EER plots for a list of experiments. It generates two ``metrics`` outputs with ERR
and HTER criteria along with `roc`, `det`, `epc`, `hist` plots for each and HTER criteria along with ``roc``, ``det``, ``epc``, ``hist`` plots for each
experiment. For example: experiment. For example:
.. code-block:: sh .. code-block:: sh
$bob measure evaluate -t -l 'my_metrics.txt' -o 'my_plots.pdf' {sys1, sys2}/ $bob measure evaluate -l 'my_metrics.txt' -o 'my_plots.pdf' {sys1, sys2}/
{eval,dev} {eval,dev}
will output metrics and plots for the two experiments (dev and eval pairs) in will output metrics and plots for the two experiments (dev and eval pairs) in
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment