script.rst



Platform Overview
Script of a video explaining platform core features:

Preparations and Preconditions

The user must not be logged in
The tutorial starts from a platform with some experiments already executed
The tutorial user must have a private experiment on eigenfaces to show the
sharing features


Script


Say: The BEAT platform is an European computing e-infrastructure for
Open Science proposing a solution for open access, scientific information
sharing and re-use including data and source code while protecting privacy
and confidentiality. It allows easy online access to experimentation and
testing in computational science. You define what data and modules you would
like to use, we make sure the system runs and provides you with a result.
Data from different experiments can be easily compared and searched. The
platform also provides an attestation mechanism for your reports (scientific
papers, technical documents or certifications). This video is a brief
beginners tutorial on some of its features.

Say: The BEAT platform runs experiments on your behalf, based on the
configuration you pick. To view all available experiments, simply click on
the button drop down on the top right corner of the page and select
"Experiments"
Action: Click on the dropdown button and select "Experiments"
Outcome: The list of public experiments is shown


Say: Here, you have a list of public experiments made available by users
of the platform, for different purposes. Let's look at a "Face Recognition
Toy-Example using Eigenfaces". To do that, use the search box located on the
top left of the experiment page and search for "eigenface"
Action: Type "tutorial/eigenface" on the experiment search text bar
Outcome: The "tutorial/eigenface" experiments are show.


Say: Let's choose one of the experiments and have a look at it.
Action: Choose one and click on it
Outcome: The experiment results are shown


Say: Here you can see the results of this experiment. The results are
defined by the users and can be composed of simple scalars (for example,
error rates) or plots. If you click on the tab that says "Execution
Details", you'll see more information about the experiment itself.
Action: Click on the tab that says "Execution Details"
Outcome: The execution details for the experiment are shown


Say: Experiments are composed of a toolchain, that describes the
dataflow in the experiment; one or more datasets, that define what is the
input data for the experiment and one or more algorithms that crunch the
dataset data in order to produce a result
Action: Show the toolchain on the right with the mouse
Action: Click on the dataset block
Outcome: the dataset block is uncollapsed
Action: Click on the dataset block
Outcome: the dataset block is collapsed
Action: Click on an algorithm block
Outcome: the algorithm block is uncollaped


Say: Generally, it is not very useful to see the results of a single
experiment. In the BEAT platform, you can search for aggregated results of
similar experiments by clicking on the search icon on the action bar.
Action: Click on the search action button
Outcome: A search result is displayed showing all relevant experiments


Say: You can now see all experiments conducted in the platform which use
the same toolchain, the same input datasets and provide the same analysis
figures. These experiments are comparable. Results are tabulated on the
bottom of the page, so you can compare your reference experiment with other
techniques to achieve the same goal.
Action: Scroll down and show results


Say: You can equally compare plots generated from these experiments. For
example, if I'd like to plot the aggregated ROC curves for all experiments,
I can just select "roc" on the display text bar to add more comparison
fields.
Action: Go to the result text bar and type "roc"
Outcome: A ROC plot is show for all experiments


Say: You may also order the table using any of the headers available.
For example, try to order the table by equal error rate or eer by clicking
on the "eer" column.
Action: Click on the eer column
Outcome: The search results are organized by eer


Say: If you're logged in, you may save the search so you can repeat it
later.

Say: Some actions on the platform such as running experiments or
storing search terms require you're logged in. If you have an account on
the BEAT platform, you can log-in using the "sign-in" button on the top
right.
Action: Point to the sign-in button


Say: If you don't have an account yet, you may create one filling the
form through the "sign-up" button. It's free.
Action: Point to the sign-up button


Say: Let's log in and revisit the experiments list.
Action: Sign-in
Action: Go to the experiments tab


Say: As you can see, more experiments are now displayed. In the BEAT
platform, the visibility of experiments, algorithms, and other objects can
be fine-tunned by the user. By default, all interaction is kept private to
you until you say otherwise.

Say: Private experiments are show with an icon displaying a single
user.  Experiments that are shared with you by other people, but are not
public are shown using an icon with more users. Public experiments are
shown with a globe.

Say: Let's revisit the experiment we visited earlier
Action: Choose the same experiment
Outcome: The experiment results are displayed


Say: As you can see, now we have an extra tab that displays sharing
properties of the experiment.
Action: Point to the "sharing" tab and click on it
Outcome: The sharing properties are displayed


Say: If this experiment would be private, you'd be able to share it
using this tab.

Say: Let's now re-execute the search.
Action: Click on the "search" action button
The search is displayed


Say: Because I'm now logged in, I can save this search for later
execution.
Action: Click on the "save" action button
Outcome: A pop-up shows up indicating the search name


Say: Simply type a name for your search and click on "Save"
Action: Type "atnt-eigenfaces"
Action: Click on the "Save" button
Outcome: The search is saved


Say: As any other object on the platform, the searches can be also
shared.  Right now, it is private, but you can change that by visiting the
"Sharing" tab as with experiments.

Say: The search is dynamic. If you reload this page, new experiments
matching your criteria will be displayed.

Say: You can optionally enable a "leaderboard". A leaderboard is a
stored search that is automatically updated every few hours. If there is a
change on the ranking, you get notified via e-mail. To enable this feature,
just click on the "leaderboard" checkbox on the "Configurations" panel.
Action: Click on the "Configuration" panel to uncollapse it
Action: Click on the "leaderboard" checkbox
Action: Click on "Save"


Say: As you go using the platform, you may want to attest experiments
so you can show them to a paper reviewer. When you attest and experiment,
the BEAT platform guarantees its reproducibility by making all of its
components unmodifiable.

Say: There are two basic ways to achieve this: using attestations or
reports.  Attestations allow you to certify a single experiment, reports
allow you to certify multiple experiments together. To attest a single
experiment, browse to the experiment page and click the medal button on the
action bar.
Action: Click on one of the unattested experiments
Outcome: The experiment is displayed
Action: Click on the attestation button on the action bar
Outcome: A pop-up is displayed showing the experiment is going to be
frozen


Say: If you click on "Attest", then the experiment will be locked by
the platform and a unique URL created to display its results. Don't worry!
The experiment remains private, but the URL allows a reviewer to check the
experiment really gives out the results you promised on the paper. The
reviewer won't have access to experiment details if it is kept private.
Action: Click on attest
Outcome: The experiment is attested


Say: Now the experiment is locked, that is, partially attested. After
your paper is published, you can come back to the platform and emit the
final certificate which will make your toolchain and experiment public. You
can optionally choose not to open-source your algorithms, in which case the
platform will make them available as a "black-box" to other users. No
access to the source-code will be possible.
Action: Click on the badge and go through the unlocking
Outcome: The experiment is now public


Say: You can also do the same with reports. Reports are like
macro-attestations as they freeze multiple experiments together. In doing
so, you can also show aggregated experiment figures and plots, like in a
search.
Action: Click on the "reports" dropdown button
Outcome: A list of reports are shown


Say: I have previously prepared a report with same experiments just as
an illustrative example.
Action: Click on the "tutorial/eigenfaces" report
Outcome: The eigenfaces report is shown


Say: This report contains a table showing the performance of the
selected experiments and a plot. I can add as many tables and plots as I
want. I just have to hit "Save" everytime I'd like to save a modification.
Action: Show the save button with the mouse


Say: As with attestations, reports provide a unique link that can be
used for reviews.
Action: Click on the numbered link in the report page
Outcome: A "reviewer" view is shown


Say: This is what a reviewer will see when you paste the report URL on
your paper.

Say: You can work with the BEAT platform in many ways. For example, you
can create searches and leaderboards that express your interest. You can
also re-run experiments easily and change parameters so to test for new
settings.

Say: For example, let's re-run the experiment with 77 components and
check if, with 50 components how it would work. Let's click on the
experiment link here on the report and go to the experiment page.
Action: Go back on the browser
Outcome: The user report page is shown
Action: Click on the "configuration" panel to uncollapse it
Outcome: The configuration panel is uncollapsed
Action: Click on the eigenfaces experiment with 77 components
Outcome: The experiment page is shown.


Say: To re-run an experiment, just click on the "Fork" button located
on the top-right action bar. When you do that, the platform copies the
whole experiment configuration to your work area. Of course that experiment
will be private to you.
Action: Click on the "fork" action button
Outcome: The platform forks the experiment and puts you in the experiment
page


Say: Now we can change any experiment parameter of our interest. In
this case, I will change the number of components to 50, select an
experiment name that I like and click on the "Go" button to re-run the
experiment.
Action: Choose 50 components
Action: Type "eigenfaces-with-50-components" on the experiment name text
bar
Action: Click on the Go button
Outcome: The experiment starts executing


Say: Once you click the go button, the experiment is submitted into the
backend processing farm and will run until it is fully executed. The
experiment page will display an animated progress log of the experiment
execution. Results are displayed back to the user when they are ready.
Action: Wait for the experiment to finish
Outcome: A yellow bar showing the experiment is finished is displayed
Action: Click on the reload link
Outcome: The experiment results are shown


Say: In a few seconds, I was able to re-run an experiment with the
parameterization I chose. Of course, I can re-run the search and compare
the performance.
Action: Click on the search action button
Outcome: The search page is loaded
Action: Click on the EER column to re-order results
Outcome: The table is re-ordered
Action: Show the relative position of the new experiment


Say: You can change more more than the parameters of a experiment. You
can actually change all aspects of it: the toolchain, the datasets and the
actual algorithms used to crunch the data. It is all up to you. You can
inspect these components by looking at the different tabs available through
the pop-down button on the top
Action: Click on the pop-down button to show the objects available
Outcome: The pop-down menu is displayed


Say: You have made your first steps into the BEAT platform. In order to
learn more, please consult our user guide by clicking on the "book" button
on the top-right corner of the page. You'll find detailed instructions
about all you can do with the platform.

Say: In case you have questions, suggestions or found bugs, please
don't hesitate in posting a message to our development mailing list at
google groups, called "beat-devel". Thanks for your interest. Keep posted
for new features and bug fixes.


Screen shows credits:
Authors
EU funding
beat-devel mailing list