Commit ca6dd12c authored by Manuel Günther's avatar Manuel Günther
Browse files

New README and documentation strategy

parent 704029a2
*~
*.swp
*.pyc
*.so
*.dylib
bin
eggs
parts
.installed.cfg
.mr.developer.cfg
*.egg-info
src
develop-eggs
dist
sphinx
dist
.nfs*
.gdb_history
build
*.egg
src/
......@@ -6,6 +6,7 @@ matrix:
env:
- secure: qL1i5WsXhScWH6AT3bThx17Puw90+w9vKVxS0fVRTo2HOQ4dpofZEnwFia+QBg5ARSiELE7Yl1kyHXyFWT4RcES0TJP5KYqhDL4LToYwKHzmlizWHN7cTPWF24fs8M06IMjhvHr0TL6VKh3rtFHhGDV6hjCNiB3OShupnQtnVN4=
- secure: V0v5m68Ed8LQufe421JwKobtVw3EE2iKbAR2JhNzwCWjej16qc4olPN3g+wPXYyTNpseC98qe2DKdl7NuREgsOarfYJs//wn7te2FSFegsbNHMoHPTeDr6UY8O7qWjnqrPAssaE0dFLPhippRCGZ8ajaJOhEOO07zYkqa10eOPo=
- BOB_DOCUMENTATION_SERVER=https://www.idiap.ch/software/bob/docs/latest/bioidiap/%s/master
- python: 3.2
env:
- NUMPYSPEC===1.8.0
......@@ -15,11 +16,11 @@ matrix:
before_install:
- sudo add-apt-repository -y ppa:biometrics/bob
- sudo apt-get update -qq
- sudo apt-get install -qq --force-yes libboost-all-dev libblitz1-dev libhdf5-serial-dev libatlas-dev libatlas-base-dev liblapack-dev
- sudo apt-get install -qq --force-yes libboost-all-dev libblitz1-dev libhdf5-serial-dev libatlas-dev libatlas-base-dev liblapack-dev texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended
- if [ -n "${NUMPYSPEC}" ]; then sudo apt-get install -qq gfortran; fi
- if [ -n "${NUMPYSPEC}" ]; then pip install --upgrade pip setuptools; fi
- if [ -n "${NUMPYSPEC}" ]; then pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel numpy$NUMPYSPEC ; fi
- pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel matplotlib==1.3.0 sphinx nose==1.3.0 jinja2==2.6 coveralls
- if [ -n "${NUMPYSPEC}" ]; then pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel numpy$NUMPYSPEC; fi
- pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel matplotlib==1.3.0 sphinx nose==1.3.0 jinja2==2.6 coveralls setuptools==7.0
install:
- python bootstrap.py
- ./bin/buildout
......
.. vim: set fileencoding=utf-8 :
.. Manuel Guenther <manuel.guenther@idiap.ch>
.. Thu Sep 4 11:35:05 CEST 2014
.. Fri Oct 31 14:18:57 CET 2014
.. image:: http://img.shields.io/badge/docs-stable-yellow.png
:target: http://pythonhosted.org/bob.db.mnist/index.html
.. image:: http://img.shields.io/badge/docs-latest-orange.png
:target: https://www.idiap.ch/software/bob/docs/latest/bioidiap/bob.db.mnist/master/index.html
.. image:: https://travis-ci.org/bioidiap/bob.db.mnist.svg?branch=master
:target: https://travis-ci.org/bioidiap/bob.db.mnist
.. image:: https://coveralls.io/repos/bioidiap/bob.db.mnist/badge.png
:target: https://coveralls.io/r/bioidiap/bob.db.mnist
.. image:: http://img.shields.io/github/tag/bioidiap/bob.db.mnist.png
:target: https://github.com/bioidiap/bob.db.mnist
.. image:: https://img.shields.io/badge/github-master-0000c0.png
:target: https://github.com/bioidiap/bob.db.mnist/tree/master
.. image:: http://img.shields.io/pypi/v/bob.db.mnist.png
:target: https://pypi.python.org/pypi/bob.db.mnist
.. image:: http://img.shields.io/pypi/dm/bob.db.mnist.png
:target: https://pypi.python.org/pypi/bob.db.mnist
.. image:: https://img.shields.io/badge/original-data--files-a000a0.png
:target: http://yann.lecun.com/exdb/mnist
================
MNIST Database
================
==================================
MNIST Database Interface for Bob
==================================
The MNIST database is a database of handwritten digits, which consists of a
training set of 60,000 examples, and a test set of 10,000 examples. It was
made available by Yann Le Cun and Corinna Cortes (`MNIST database
<http://yann.lecun.com/exdb/mnist/>`_). The data was originally extracted
from a larger set made available by `NIST <http://www.nist.gov/>`_, before
being size-normalized and centered in a fixed-size image (28x28 pixels).
The MNIST_ database is a database of handwritten digits, which consists of a training set of 60,000 examples, and a test set of 10,000 examples.
It was made available by Yann Le Cun and Corinna Cortes.
The data was originally extracted from a larger set made available by NIST_, before being size-normalized and centered in a fixed-size image (28x28 pixels).
The actual raw data for the database should be downloaded from the `original
website <http://yann.lecun.com/exdb/mnist/>`_. This package only contains
the `Bob <http://www.idiap.ch/software/bob/>`_ accessor methods to use this
database directly from python.
This package only contains the Bob_ accessor methods to use this database directly from Python.
It does not contain the original raw data files, which need to be obtained through the link above.
You would normally not install this package unless you are maintaining it. What
you would do instead is to tie it in at the package you need to **use** it.
There are a few ways to achieve this:
Installation
------------
To install this package -- alone or together with other `Packages of Bob <https://github.com/idiap/bob/wiki/Packages>`_ -- please read the `Installation Instructions <https://github.com/idiap/bob/wiki/Installation>`_.
For Bob_ to be able to work properly, some dependent packages are required to be installed.
Please make sure that you have read the `Dependencies <https://github.com/idiap/bob/wiki/Dependencies>`_ for your operating system.
1. You can add this package as a requirement at the ``setup.py`` for your own
`satellite package
<https://github.com/idiap/bob/wiki/Virtual-Work-Environments-with-Buildout>`_
or to your Buildout ``.cfg`` file, if you prefer it that way. With this
method, this package gets automatically downloaded and installed on your
working environment, or
Documentation
-------------
For further documentation on this package, please read the `Stable Version <http://pythonhosted.org/bob.db.mnist/index.html>`_ or the `Latest Version <https://www.idiap.ch/software/bob/docs/latest/bioidiap/bob.db.mnist/master/index.html>`_ of the documentation.
For a list of tutorials on this or the other packages ob Bob_, or information on submitting issues, asking questions and starting discussions, please visit its website.
2. You can manually download and install this package using commands like
``easy_install`` or ``pip``.
.. _bob: https://www.idiap.ch/software/bob
.. _mnist: http://yann.lecun.com/exdb/mnist
.. _nist: http://www.nist.gov
The package is available in two different distribution formats:
1. You can download it from `PyPI <http://pypi.python.org/pypi/bob.db.mnist>`_, or
2. You can download it in its source form from `its git repository
<https://github.com/bioidiap/bob.db.mnist>`_.
The database raw files must be installed somewhere in your environment.
You can mix and match points 1/2 above based on your requirements. Here
are some examples:
Modify your setup.py and download from PyPI
===========================================
That is the easiest. Edit your ``setup.py`` in your satellite package and add
the following entry in the ``install_requires`` section (note: ``...`` means
`whatever extra stuff you may have in-between`, don't put that on your
script)::
install_requires=[
...
"bob.db.mnist",
],
Proceed normally with your ``bootstrap/buildout`` steps and you should be all
set. That means you can now import the namespace ``bob.db.mnist`` into your scripts.
Modify your buildout.cfg and download from git
==============================================
You will need to add a dependence to `mr.developer
<http://pypi.python.org/pypi/mr.developer/>`_ to be able to install from our
git repositories. Your ``buildout.cfg`` file should contain the following
lines::
[buildout]
...
extensions = mr.developer
auto-checkout = *
eggs = bob
...
bob.db.mnist
[sources]
bob.db.mnist = git https://github.com/bioidiap/bob.db.mnist.git
...
How to use this database API
============================
After launching the python interpreter (assuming that the environment is properly set up),
you could get the training set as follows::
>>> import bob.db.mnist
>>> db = bob.db.mnist.Database('PATH_TO_DATA_FROM_YANN_LECUN_WEBSITE') # 4 binary .gz compressed files
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
In this case, this should return two NumPy arrays:
1. `images` contain the raw data (60,000 samples of dimension 784 [28x28 pixels images])
2. `labels` are the corresponding classes (digits 0 to 9) for each of the 60,000 samples
If you don't have the data installed on your machine, you can also use the following
set of commands that will:
1. first look for the database in the bob/db/mnist/ subdirectory and use it if is available
2. automatically download it from Yann Lecun's website into a temporary folder that will
be erased when the destructor of the bob.db.mnist database is called.
3. automatically download it into the provided directory that will **not** be deleted.
::
>>> import bob.db.mnist
>>> db = bob.db.mnist.Database() # Check for the data files locally, and download them if required
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
>>> del db # delete the temporary downloaded files if any
or
::
>>> db = bob.db.mnist.Database("Directory") # Persistently downloads files into the folder "Directory"
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
>>> del db # The download directory stays
......@@ -3,21 +3,13 @@
# Laurent El Shafey <Laurent.El-Shafey@idiap.ch>
# Wed May 8 19:05:23 CEST 2013
#
# Copyright (C) 2011-2013 Idiap Research Institute, Martigny, Switzerland
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Copyright (C) 2011-2014 Idiap Research Institute, Martigny, Switzerland
import os
import sys
import glob
import pkg_resources
import sys, os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
......@@ -71,15 +63,17 @@ project = u'MNIST Database (Bob API)'
import time
copyright = u'%s, Idiap Research Institute' % time.strftime('%Y')
# Grab the setup entry
distribution = pkg_resources.require('bob.db.mnist')[0]
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
from bob.db.mnist.driver import Interface
version = Interface().version()
version = distribution.version
# The full version, including alpha/beta/rc tags.
release = version
release = distribution.version
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
......@@ -140,12 +134,12 @@ if sphinx.__version__ >= "1.0":
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = ''
html_logo = 'img/logo.png'
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
html_favicon = ''
html_favicon = 'img/favicon.ico'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
......@@ -194,7 +188,7 @@ html_favicon = ''
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'bobdbdoc'
htmlhelp_basename = 'bob_db_mnist_doc'
# -- Options for LaTeX output --------------------------------------------------
......@@ -208,7 +202,7 @@ latex_font_size = '10pt'
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'bobdbman.tex', u'Bob',
('index', 'bob_db_mnist.tex', u'Bob',
u'Biometrics Group, Idiap Research Institute', 'manual'),
]
......@@ -246,40 +240,15 @@ man_pages = [
('index', 'bob', u'MNIST Database (Bob API) Documentation', [u'Idiap Research Institute'], 1)
]
# We want to remove all private (i.e. _. or __.__) members
# that are not in the list of accepted functions
accepted_private_functions = ['__call__']
def member_function_test(app, what, name, obj, skip, options):
# test if we have a private function
if len(name) > 1 and name[0] == '_':
# test if this private function should be allowed
if name not in accepted_private_functions:
# omit privat functions that are not in the list of accepted private functions
return True
else:
# test if the method is documented
if not hasattr(obj, '__doc__') or not obj.__doc__:
return True
# Skips selected members in auto-generated documentation. Unfortunately, old
# versions of Boost.Python will not generate a __self__ member for static
# methods and that screws-up Sphinx processing.
if sphinx.__version__ < "1.0":
# We have to remove objects that do not have a __self__ attribute set
import types
if isinstance(obj, types.BuiltinFunctionType) and \
not hasattr(obj, '__self__') and what == 'class':
app.warn("Skipping %s %s (no __self__)" % (what, name))
return True
return False
# Default processing flags for sphinx
autoclass_content = 'both'
autodoc_member_order = 'bysource'
autodoc_default_flags = ['members', 'undoc-members', 'special-members', 'inherited-members', 'show-inheritance']
autodoc_default_flags = ['members', 'undoc-members', 'inherited-members', 'show-inheritance']
# For inter-documentation mapping:
from bob.extension.utils import link_documentation
intersphinx_mapping = link_documentation()
def setup(app):
app.connect('autodoc-skip-member', member_function_test)
pass
......@@ -6,5 +6,43 @@
User's Guide
==============
After launching the python interpreter (assuming that the environment is properly set up), you could get the training set as follows:
.. code-block:: py
>>> import bob.db.mnist
>>> db = bob.db.mnist.Database('PATH_TO_DATA_FROM_YANN_LECUN_WEBSITE') # 4 binary .gz compressed files
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
In this case, this should return two :py:class:`numpy.ndarray`\s:
1. `images` contain the raw data (60,000 samples of dimension 784 [28x28 pixels images])
2. `labels` are the corresponding classes (digits 0 to 9) for each of the 60,000 samples
If you don't have the data installed on your machine, you can also use the following set of commands that will:
1. first look for the database in the ``bob/db/mnist`` subdirectory and use it if is available
2. automatically download it from Yann Lecun's website into a temporary folder that will be erased when the destructor of the :py:class:`bob.db.mnist.Database` is called.
3. automatically download it into the provided directory that will **not** be deleted.
.. code-block:: py
>>> import bob.db.mnist
>>> db = bob.db.mnist.Database() # Check for the data files locally, and download them if required
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
>>> del db # delete the temporary downloaded files if any
or:
.. code-block:: py
>>> db = bob.db.mnist.Database("Directory") # Persistently downloads files into the folder "Directory"
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
>>> del db # The download directory stays
.. todo::
Write users guide.
......@@ -2,8 +2,10 @@
.. @author: Manuel Guenther <Manuel.Guenther@idiap.ch>
.. @date: Tue Aug 26 09:42:18 CEST 2014
.. _bob.db.mnist:
==========================
MNIST Database Protocols
MNIST Database Interface
==========================
.. todolist::
......
......@@ -48,15 +48,14 @@ setup(
],
entry_points={
# declare database to bob
'bob.db': [
'mnist = bob.db.mnist.driver:Interface',
],
},
classifiers = [
'Framework :: Bob',
'Development Status :: 4 - Beta',
'Environment :: Console',
'Intended Audience :: Developers',
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment