Plotting library request (migrated from github)

It would be useful if the platform provided a set of library functions to allow the creation of charts in easier terms, instead of everyone re-implementing theirs. Bob already provides APIs for plotting. It would make sense that BEAT provided a similar API for functionality in Bob:

A good starting point for this API is Bob's plotting API, which is available here.

A tricky plot to perform is a DET curve. I hope this is still possible using RGraph's API.

In particular, ISO/IEC 19795-1:2006 "Biometric performance testing and reporting -- Part 1: Principles and framework" (from page 33 until page 37), establishes the correct set of plots and notations to deploy for performance reporting under open/closed-set identification and recognition.

It is not only the rename FAR/FRR as FNMR/FMR but one of the tricky aspect is also to track and report Failure to Capture and to Enroll if we want to plot FAR/FRR and FNMR/FMR separately !

The matter

1. At present, we need to define, for every type of plot supported, an equivalent dataformat. Today, we have a dataformat only for a single line plot, with linear axis.
2. How to visualize this line plot is hardcoded into beat.web, as it has to bridge the plot description with the RGraph (javascript) plotting library. The user has control on certain plot aspects, such as gutter sizes and labels, but not fundamental control on RGraph itself so it can do whatever he/she wants.

Some thinking

1. The definition of formats for plotting seems to be a needed restriction: we cannot just allow the user to plot no matter what given the platform data restrictions.
2. It would be nice to allow the user to contribute visualisation routines to the platform, like with any other contribution. Maybe a good compromise would be by inserting the visualizations in a separate package and allowing the user to play with them, improving them.

Action plan

1. We should separate the definition of dataformats and visualisations into a external beat.plot package, fully testable by itself.

2. At beat.web, we should use a certain version of this package, which maps existing plot types (charts, histograms, scatter plots, etc), into visualisations. Several visualisations may be applicable to a given dataformat. For example, a beat/chart may be plotted as a line figure or a scatter plot.

3. The work flow is like this:

   1. The user makes a search query on the BEAT website
   2. The user filters the result until a single analysis algorithm is used for all results displayed
   3. The platform allows the user to choose the visualisation type for each applicable result
   4. For each visualisation, the user may apply different options defined on that visualisation (e.g., change colors, line type, modify axis labels, etc).

Potential issues

1. Once a given visualisation is installed at the platform, we cannot change it anymore as it may affect stored searches. A solution to this problem would be to version visualisations, pretty much like we do for other contributions in the system.

After talking to Christoph today, please note we must use FMR/FNMR. He does not advise to use the other option.

Another point is that a log-scale x-axis is preferred for ROC curves (and maybe also for CMC curves). Unfortunately, I couldn't figure out, how to display the x-axis in logarithmic scale using the current plotting tool.

In fact, the trick I tried was to simply compute the log() of my data. Plotting this results in the correct plot, but there is no (simple) way to define the x-labels correctly. The only solution for writing custom x-tick labels is to call a (I guess Java-script) function to turn a value into a label: http://www.rgraph.net/docs/scatter.html#custom.tickmarks I have no clue, how to provide such a function in our python interface, though.

I would very much appreciate if we could use another plotting tool that RGraph. One such possibility would be matplotlib, but also tools like GnuPlot might be interesting since it has back-ends for several programming languages. Users could either provide their own gnuplot plotting script, or -- due to data restrictions -- be allowed to post-process options like axis labels.

This is a pending issue - I'm not sure how to deal with this best. A rewrite of the plotting support may be a project of its own.

I've checked RGraph and I confirm it does not support log axes.

The more I see this and talk to people, the more I get convinced that it is not a good option to make users plot anything using their analyzers. The analyzer should just dump the data and announce what the data is. For example: here is the x and y data for a scatter plot or here is the x and y data for an histogram. It should be the job of the "search page results" to allow the user to format the plot, add axis titles, captions, etc, to the plot.

This strategy moves the formatting of a plot to the moment it needs to be finalized and gives the beat.web package a bit more of flexibility to deciding how to draw or export plot data for/to the user.

mentioned in commit 6e241c18

Status changed to closed

This is now implemented this way. RGraph is out of the equation for the experiment plots. It's kept in the system though for other plots.