Skip to content
Snippets Groups Projects
Commit e8a83f9b authored by Jaden's avatar Jaden
Browse files

update user documentation for the reports changes, #473

parent 1e94fed0
No related branches found
No related tags found
1 merge request!225Update user documentation for the reports changes
Pipeline #
......@@ -97,16 +97,6 @@ It is possible to add experiments to *existing* reports in a number of ways:
* from the results of a :ref:`searchlabel`.
.. important::
In order to properly compare experiments, the single requirement built-into
the platform is that they use the same *analysis* algorithm (i.e., the
algorithm that is placed on the last block of a toolchain).
This is checked by the platform when you try to add an experiment that does
not match this criteria.
In order to add an experiment to a report, click on the check boxes to the left
of the relevant experiment lists and then click on the button "Add to report",
situated on the top of this list. A pop-up will then appear asking you to
......@@ -115,6 +105,11 @@ and then confirm using the button "Add". Another pop-up will appear indicating
how many experiments, out of the selected ones, were successfuly added to the
report.
You may also a list of experiments through the search tab when. (Note that you
can add experiments to your report from a search list too).
.. image:: img/figure_2_analyzer_search_report.*
.. note::
......@@ -124,6 +119,25 @@ report.
If an experiment was already added to a report, re-adding it does **not**
duplicate the experiment on the report.
Creating & Managing Report Groups
=================================
Once you have experiments in a report, you must create at least one report
`group`. Each group holds a subset of the report's experiments and compares the
experiments' results. In these groups you may add figures and tables, as well
as blocks of text.
.. important::
In order to properly compare experiments, the single requirement built into
the platform is that each experiment in a group uses the same *analysis*
algorithm (i.e., the algorithm that is placed on the last block of a
toolchain).
This is checked by the platform when you try to add an experiment to a group
that does not match this criteria. If the group has no experiments, you may
add any experiment.
Report states
=============
......@@ -137,15 +151,19 @@ A report could be in one of these three states:
* You can add documentation. Use this feature as a *caption* to your figure
and tables, explaining the reader what is displayed.
* You can remove experiments from the report
* You can set-up aliases to experiments. Very often, when one publishes
content from experiments executed, it is nice to be able to reset
experiment names so as to identify specific use conditions. The platform
allows this through the aliasing feature. You can setup an experiment
alias, for example, to say *"our system"*, distinguishing your
contribution from the other experiments in the report.
* You can add or delete report items (i.e. tables and figures)
* You can set-up aliases to experiments in each report group that it
belongs. Very often, when one publishes content from experiments executed,
it is nice to be able to reset experiment names so as to identify specific
use conditions. The platform allows this through the aliasing feature. You
can setup an experiment alias, for example, to say *"our system"*,
distinguishing your contribution from the other experiments in the report
group (and, thusly, in the group's figures and tables).
* You can add, delete, rename, and reorganize report groups and their items
(i.e. tables, figures, and text blocks)
* The floating point precision could be set to fix the precision of floating
point outputs in your tables
* The specific plotter configuration may be changed for plots, if there are
multiple valid configurations available.
2. **Locked**: This state is intended for review purposes. Once flipped,
reviewers have the guarantee that your results are validated by the
......@@ -197,81 +215,65 @@ URL.
Adding tables and plots
=======================
Adding tables and charts can be easily done with the help of the *Add a report
item* button. A pop up menu will ask you to select if you wish to add a table
or a figure.
Adding tables, charts, and text blocks can be easily done with the help of the
*Add Plot*, *Add Table*, and *Add Text Block* buttons in the header of each
group in the report. The group must have at least 1 experiment in it to add
report items.
According to the choice you have made, you will be able to select the type of
figure (for a plot) or the information you wish to compare (for a table), among
selected experiments.
You will be able to select the type of figure (for a plot) or the information
you wish to compare (for a table), among the group's experiments.
Exporting a figure
==================
Any figure from your report can be exported to PNG, JPEG or PDF format using
the corresponding "Export Figure" button from the figure panel.
the corresponding "Download" button from the figure panel.
Exporting a table
=================
Any table from your report can be exported to a CSV format using the "Export
Table" button from the table panel.
Any table from your report can be shown as copy-friendly CSV text by using the
"Toggle CSV" button from the table panel.
Export a text block
===================
You can see the source (ReStructured Text format) of any text block by using
the "Toggle Source View" button from the text block panel.
Comparing results
=================
The |project| platform allows you to compare results of experiments together
creating aggregated tables and figures. The only requirement which is imposed
by the platform is that experiments must use the same analyzer algorithm. This
requirements simplifies the comparison machinery and is enforced by the report
application.
Comparing Plot Results
~~~~~~~~~~~~~~~~~~~~~~
If a report can't add experiments due to this reason, you'll be warned in a
visible manner, such as displayed in the figure below:
.. image:: img/figure_1_analyzer_mismatch_report.*
In this case, make sure to select the appropriate experiments with comparable
output analyzers. You could check the inputs and outputs from your experiments
or narrow a list of experiments with this criteria throught the search tab when
using corresponding filters. (Note that you can add experiments to your report
from a search list too)
.. image:: img/figure_2_analyzer_search_report.*
Comparing results from a plot: Depending on the type of plot, the user is able
to select the "plotter" and "plotter parameter" of his wish. If the graphic is
cluttered by too much of information or if you wish to plot each experiment
independently, you can easily click on the "expand button" on the specific plot
panel to display multiple plots.
Depending on the type of plot, the user is able to select the "plotter" and
"plotter parameter" as they wish.
.. image:: img/figure_3a_plot_plotter_parameters_report.*
If the graphic is cluttered by too much information or if you wish to plot
each experiment independently, you can easily click on the "Expand" button on
the specific plot panel to display multiple plots.
.. image:: img/figure_3b_plot_expand_report.*
Use the "Merge" button to combine expanded plots back into one.
Comparing Table Results
~~~~~~~~~~~~~~~~~~~~~~~
Comparing results from a table: It is possible to compare a large panel of
items on a table, including:
It is possible to compare a large panel of items on a table, including:
* results from the experiments
* execution time information for each experiment
* execution time information for similar blocks from each experiment
* parameters from various corresponding algorithms on each experiments
.. image:: img/figure_4a_table_report.*
.. image:: img/figure_4b_table_report.*
.. image:: img/figure_4c_table_report.*
You can display the column selector by clicking the "Choose Columns" button
in a table panel.
Saving your Report
------------------
......@@ -318,3 +320,38 @@ be restored. Its unique URL is lost.
.. include:: ../links.rst
Example Workflow
----------------
To tie together the above documentation, the following is an example workflow
for creating and editing a report:
#. To begin, create a report on the reports list page (as described above).
#. Add experiments to the report, either by browsing & selecting experiments
through the experiments list page or adding individual experiments from
their pages.
#. Back in your report, add a description for the report.
#. Create your report groups for comparing different experiments. For example,
creating "Performance on Database 1" (Group 1), "Performance on Database 2"
(Group 2), and "Overall Performance" (Group 3) groups for comparing
experiments that were ran using Database 1, experiments using Database 2,
and all experiments.
#. Add experiments to your report groups (remember, experiments must have the
same analyzer to be comparable!). Continuing with the above example, we
assume all the experiments in the report have the same analyzer. We add any
experiments using Database 1 to Group 1, any experiments using Database 2,
and all experiments to Group 3.
#. In each group, change your experiments' aliases (each group has its own list
of aliases).
#. In each group, add & adjust your report items. You can use text blocks to
explain plots and tables or describe groups.
#. Now, click on the link to the numbered version of your report to see it as
others will see it. Remember, to see changes from edit-mode, you must save
your changes!
#. Continue to change your report, occasionally checking the numbered version
to make sure others will see what you want. Remember, you can reorder items
inside of a group and groups inside of report as well as renaming groups &
items.
#. Eventually, you will want to lock your report, and then publish it.
doc/user/reports/img/figure_1_analyzer_mismatch_report.png

14.2 KiB

doc/user/reports/img/figure_3a_plot_plotter_parameters_report.png

56.1 KiB | W: | H:

doc/user/reports/img/figure_3a_plot_plotter_parameters_report.png

60.7 KiB | W: | H:

doc/user/reports/img/figure_3a_plot_plotter_parameters_report.png
doc/user/reports/img/figure_3a_plot_plotter_parameters_report.png
doc/user/reports/img/figure_3a_plot_plotter_parameters_report.png
doc/user/reports/img/figure_3a_plot_plotter_parameters_report.png
  • 2-up
  • Swipe
  • Onion skin
doc/user/reports/img/figure_3b_plot_expand_report.png

60.3 KiB | W: | H:

doc/user/reports/img/figure_3b_plot_expand_report.png

68.4 KiB | W: | H:

doc/user/reports/img/figure_3b_plot_expand_report.png
doc/user/reports/img/figure_3b_plot_expand_report.png
doc/user/reports/img/figure_3b_plot_expand_report.png
doc/user/reports/img/figure_3b_plot_expand_report.png
  • 2-up
  • Swipe
  • Onion skin
doc/user/reports/img/figure_4a_table_report.png

18.4 KiB

doc/user/reports/img/figure_4b_table_report.png

44.4 KiB

doc/user/reports/img/figure_4c_table_report.png

22.4 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment