From da283be63e5810c1a8b32f464bcff9cc28af3bf6 Mon Sep 17 00:00:00 2001
From: Tim Laibacher <tim.laibacher@idiap.ch>
Date: Tue, 10 Sep 2019 10:50:56 +0200
Subject: [PATCH] Add bibtex and additional material
---
README.rst | 18 ++++++++++++++
doc/covdresults.rst | 59 +++++++++++++++++++++++++--------------------
doc/index.rst | 19 +++++++++++++++
3 files changed, 70 insertions(+), 26 deletions(-)
diff --git a/README.rst b/README.rst
index 95e9e7f7..d846e596 100644
--- a/README.rst
+++ b/README.rst
@@ -30,6 +30,24 @@ package, run::
$ conda install bob.ip.binseg
+Citation
+--------
+Please use the below BibTeX reference to cite this work::
+
+ @ARTICLE{laibacher_anjos_2019,
+ author = {{Laibacher}, Tim and {Anjos}, Andr{\'e}},
+ title = "{On the Evaluation and Real-World Usage Scenarios of Deep Vessel Segmentation for Funduscopy}",
+ journal = {arXiv e-prints},
+ keywords = {Computer Science - Computer Vision and Pattern Recognition},
+ year = "2019",
+ month = "Sep",
+ url = {https://arxiv.org/abs/1909.03856},
+ eid = {arXiv:1909.03856},
+ pages = {arXiv:1909.03856},
+ archivePrefix = {arXiv},
+ eprint = {1909.03856}}
+
+
Contact
-------
diff --git a/doc/covdresults.rst b/doc/covdresults.rst
index 95f7d7fe..bf2305f4 100644
--- a/doc/covdresults.rst
+++ b/doc/covdresults.rst
@@ -6,38 +6,45 @@
COVD- and COVD-SLL Results
==========================
+In addition to the M2U-Net architecture, we also evaluated the larger DRIU network and a variation of it
+that contains batch normalization (DRIU BN) on COVD- and COVD-SSL. Perhaps surprisingly, for the
+majority of combinations, the performance of the DRIU variants are roughly equal or worse than the M2U-Net.
+We anticipate that one reason for this could be overparameterization of large VGG16 models
+that are pretrained on ImageNet.
+
F1 Scores
===========
-F1 score together with standard deviation across test images.
-
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| F1 score | :ref:`bob.ip.binseg.configs.models.driu` | :ref:`bob.ip.binseg.configs.models.driubn` | :ref:`bob.ip.binseg.configs.models.m2unet` |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-drive` | 0.788 (0.018) | 0.797 (0.019) | `0.789 (0.018) <m2unet_covd-drive.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-drive_ssl` | 0.785 (0.018) | 0.783 (0.019) | `0.791 (0.014) <m2unet_covd-drive_ssl.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-stare` | 0.778 (0.117) | 0.778 (0.122) | `0.812 (0.046) <m2unet_covd-stare.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-stare_ssl` | 0.788 (0.102) | 0.811 (0.074) | `0.820 (0.044) <m2unet_covd-stare_ssl.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1` | 0.796 (0.027) | 0.791 (0.025) | `0.788 (0.024) <m2unet_covd-chasedb1.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1_ssl` | 0.796 (0.024) | 0.798 (0.025) | `0.799 (0.026) <m2unet_covd-chasedb1_ssl.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-hrf` | 0.799 (0.044) | 0.800 (0.045) | `0.802 (0.045) <m2unet_covd-hrf.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-hrf_ssl` | 0.799 (0.044) | 0.784 (0.048) | `0.797 (0.044) <m2unet_covd-hrf_ssl.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-iostar` | 0.791 (0.021) | 0.777 (0.032) | `0.793 (0.015) <m2unet_covd-iostar.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
-| :ref:`bob.ip.binseg.configs.datasets.covd-iostar_ssl` | 0.797 (0.017) | 0.811 (0.074) | `0.785 (0.018) <m2unet_covd-iostar_ssl.pth>`_ |
-+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
+Comparison of F1-micro-scores (std) of DRIU and M2U-Net on COVD- and COVD-SSL.
+Standard deviation across test-images in brackets.
+
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| F1 score | :ref:`bob.ip.binseg.configs.models.driu`/:ref:`bob.ip.binseg.configs.models.driussl` | :ref:`bob.ip.binseg.configs.models.driubn`/:ref:`bob.ip.binseg.configs.models.driubnssl` | :ref:`bob.ip.binseg.configs.models.m2unet`/:ref:`bob.ip.binseg.configs.models.m2unetssl` |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-drive` | 0.788 (0.018) | 0.797 (0.019) | `0.789 (0.018) <m2unet_covd-drive.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-drive_ssl` | 0.785 (0.018) | 0.783 (0.019) | `0.791 (0.014) <m2unet_covd-drive_ssl.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-stare` | 0.778 (0.117) | 0.778 (0.122) | `0.812 (0.046) <m2unet_covd-stare.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-stare_ssl` | 0.788 (0.102) | 0.811 (0.074) | `0.820 (0.044) <m2unet_covd-stare_ssl.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1` | 0.796 (0.027) | 0.791 (0.025) | `0.788 (0.024) <m2unet_covd-chasedb1.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1_ssl` | 0.796 (0.024) | 0.798 (0.025) | `0.799 (0.026) <m2unet_covd-chasedb1_ssl.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-hrf` | 0.799 (0.044) | 0.800 (0.045) | `0.802 (0.045) <m2unet_covd-hrf.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-hrf_ssl` | 0.799 (0.044) | 0.784 (0.048) | `0.797 (0.044) <m2unet_covd-hrf_ssl.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-iostar` | 0.791 (0.021) | 0.777 (0.032) | `0.793 (0.015) <m2unet_covd-iostar.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
+| :ref:`bob.ip.binseg.configs.datasets.covd-iostar_ssl` | 0.797 (0.017) | 0.811 (0.074) | `0.785 (0.018) <m2unet_covd-iostar_ssl.pth>`_ |
++---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
M2U-Net Precision vs. Recall Curves
===================================
-
+Precision vs. recall curves for each evaluated dataset.
Note that here the F1-score is calculated on a macro level (see paper for more details).
.. figure:: img/pr_CHASEDB1.png
diff --git a/doc/index.rst b/doc/index.rst
index 013e46c2..82034646 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -9,6 +9,25 @@
Package to benchmark and evaluate a range of neural network architectures for
binary segmentation tasks on 2D Eye Fundus Images (2DFI). It is build using PyTorch.
+Please use the below BibTeX reference to cite this work::
+
+ @ARTICLE{laibacher_anjos_2019,
+ author = {{Laibacher}, Tim and {Anjos}, Andr{\'e}},
+ title = "{On the Evaluation and Real-World Usage Scenarios of Deep Vessel Segmentation for Funduscopy}",
+ journal = {arXiv e-prints},
+ keywords = {Computer Science - Computer Vision and Pattern Recognition},
+ year = "2019",
+ month = "Sep",
+ url = {https://arxiv.org/abs/1909.03856},
+ eid = {arXiv:1909.03856},
+ pages = {arXiv:1909.03856},
+ archivePrefix = {arXiv},
+ eprint = {1909.03856}}
+
+Additional Material
+===================
+
+The additional material referred to in the paper can be found under :ref:`bob.ip.binseg.covdresults`
Users Guide
===========
--
GitLab