Skip to content
Snippets Groups Projects
Commit 977f62d8 authored by Tim Laibacher's avatar Tim Laibacher
Browse files

Merge branch 'references' into 'master' 

Add bibtex and additional material

See merge request bob/bob.ip.binseg!7
parents 918b580e da283be6
No related branches found
No related tags found
1 merge request!7Add bibtex and additional material
Pipeline #33176 passed
Stage: build
Stage: deploy
Hide whitespace changes
Inline Side-by-side
README.rst
+ 18
− 0
View file @ 977f62d8
......@@ -30,6 +30,24 @@ package, run::
$ conda install bob.ip.binseg
Citation
--------
Please use the below BibTeX reference to cite this work::
@ARTICLE{laibacher_anjos_2019,
author = {{Laibacher}, Tim and {Anjos}, Andr{\'e}},
title = "{On the Evaluation and Real-World Usage Scenarios of Deep Vessel Segmentation for Funduscopy}",
journal = {arXiv e-prints},
keywords = {Computer Science - Computer Vision and Pattern Recognition},
year = "2019",
month = "Sep",
url = {https://arxiv.org/abs/1909.03856},
eid = {arXiv:1909.03856},
pages = {arXiv:1909.03856},
archivePrefix = {arXiv},
eprint = {1909.03856}}
Contact
-------
......
doc/covdresults.rst
+ 33
− 26
View file @ 977f62d8
......@@ -6,38 +6,45 @@
COVD- and COVD-SLL Results
==========================
In addition to the M2U-Net architecture, we also evaluated the larger DRIU network and a variation of it
that contains batch normalization (DRIU BN) on COVD- and COVD-SSL. Perhaps surprisingly, for the
majority of combinations, the performance of the DRIU variants are roughly equal or worse than the M2U-Net.
We anticipate that one reason for this could be overparameterization of large VGG16 models
that are pretrained on ImageNet.
F1 Scores
===========
F1 score together with standard deviation across test images.
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| F1 score | :ref:`bob.ip.binseg.configs.models.driu` | :ref:`bob.ip.binseg.configs.models.driubn` | :ref:`bob.ip.binseg.configs.models.m2unet` |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-drive` | 0.788 (0.018) | 0.797 (0.019) | `0.789 (0.018) <m2unet_covd-drive.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-drive_ssl` | 0.785 (0.018) | 0.783 (0.019) | `0.791 (0.014) <m2unet_covd-drive_ssl.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-stare` | 0.778 (0.117) | 0.778 (0.122) | `0.812 (0.046) <m2unet_covd-stare.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-stare_ssl` | 0.788 (0.102) | 0.811 (0.074) | `0.820 (0.044) <m2unet_covd-stare_ssl.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1` | 0.796 (0.027) | 0.791 (0.025) | `0.788 (0.024) <m2unet_covd-chasedb1.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1_ssl` | 0.796 (0.024) | 0.798 (0.025) | `0.799 (0.026) <m2unet_covd-chasedb1_ssl.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-hrf` | 0.799 (0.044) | 0.800 (0.045) | `0.802 (0.045) <m2unet_covd-hrf.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-hrf_ssl` | 0.799 (0.044) | 0.784 (0.048) | `0.797 (0.044) <m2unet_covd-hrf_ssl.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-iostar` | 0.791 (0.021) | 0.777 (0.032) | `0.793 (0.015) <m2unet_covd-iostar.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-iostar_ssl` | 0.797 (0.017) | 0.811 (0.074) | `0.785 (0.018) <m2unet_covd-iostar_ssl.pth>`_ |
+---------------------------------------------------------+---------------------------------------------+-----------------------------------------------+---------------------------------------------------+
Comparison of F1-micro-scores (std) of DRIU and M2U-Net on COVD- and COVD-SSL.
Standard deviation across test-images in brackets.
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| F1 score | :ref:`bob.ip.binseg.configs.models.driu`/:ref:`bob.ip.binseg.configs.models.driussl` | :ref:`bob.ip.binseg.configs.models.driubn`/:ref:`bob.ip.binseg.configs.models.driubnssl` | :ref:`bob.ip.binseg.configs.models.m2unet`/:ref:`bob.ip.binseg.configs.models.m2unetssl` |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-drive` | 0.788 (0.018) | 0.797 (0.019) | `0.789 (0.018) <m2unet_covd-drive.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-drive_ssl` | 0.785 (0.018) | 0.783 (0.019) | `0.791 (0.014) <m2unet_covd-drive_ssl.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-stare` | 0.778 (0.117) | 0.778 (0.122) | `0.812 (0.046) <m2unet_covd-stare.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-stare_ssl` | 0.788 (0.102) | 0.811 (0.074) | `0.820 (0.044) <m2unet_covd-stare_ssl.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1` | 0.796 (0.027) | 0.791 (0.025) | `0.788 (0.024) <m2unet_covd-chasedb1.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-chasedb1_ssl` | 0.796 (0.024) | 0.798 (0.025) | `0.799 (0.026) <m2unet_covd-chasedb1_ssl.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-hrf` | 0.799 (0.044) | 0.800 (0.045) | `0.802 (0.045) <m2unet_covd-hrf.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-hrf_ssl` | 0.799 (0.044) | 0.784 (0.048) | `0.797 (0.044) <m2unet_covd-hrf_ssl.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-iostar` | 0.791 (0.021) | 0.777 (0.032) | `0.793 (0.015) <m2unet_covd-iostar.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
| :ref:`bob.ip.binseg.configs.datasets.covd-iostar_ssl` | 0.797 (0.017) | 0.811 (0.074) | `0.785 (0.018) <m2unet_covd-iostar_ssl.pth>`_ |
+---------------------------------------------------------+--------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------+
M2U-Net Precision vs. Recall Curves
===================================
Precision vs. recall curves for each evaluated dataset.
Note that here the F1-score is calculated on a macro level (see paper for more details).
.. figure:: img/pr_CHASEDB1.png
......
doc/index.rst
+ 19
− 0
View file @ 977f62d8
......@@ -9,6 +9,25 @@
Package to benchmark and evaluate a range of neural network architectures for
binary segmentation tasks on 2D Eye Fundus Images (2DFI). It is build using PyTorch.
Please use the below BibTeX reference to cite this work::
@ARTICLE{laibacher_anjos_2019,
author = {{Laibacher}, Tim and {Anjos}, Andr{\'e}},
title = "{On the Evaluation and Real-World Usage Scenarios of Deep Vessel Segmentation for Funduscopy}",
journal = {arXiv e-prints},
keywords = {Computer Science - Computer Vision and Pattern Recognition},
year = "2019",
month = "Sep",
url = {https://arxiv.org/abs/1909.03856},
eid = {arXiv:1909.03856},
pages = {arXiv:1909.03856},
archivePrefix = {arXiv},
eprint = {1909.03856}}
Additional Material
===================
The additional material referred to in the paper can be found under :ref:`bob.ip.binseg.covdresults`
Users Guide
===========
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment