Commit 352b15d5 authored by Manuel Günther's avatar Manuel Günther
Browse files

Moved to new bob version.

parent 47443458
language: python
matrix:
include:
- python: 2.6
- python: 2.7
env:
- secure: qL1i5WsXhScWH6AT3bThx17Puw90+w9vKVxS0fVRTo2HOQ4dpofZEnwFia+QBg5ARSiELE7Yl1kyHXyFWT4RcES0TJP5KYqhDL4LToYwKHzmlizWHN7cTPWF24fs8M06IMjhvHr0TL6VKh3rtFHhGDV6hjCNiB3OShupnQtnVN4=
- secure: V0v5m68Ed8LQufe421JwKobtVw3EE2iKbAR2JhNzwCWjej16qc4olPN3g+wPXYyTNpseC98qe2DKdl7NuREgsOarfYJs//wn7te2FSFegsbNHMoHPTeDr6UY8O7qWjnqrPAssaE0dFLPhippRCGZ8ajaJOhEOO07zYkqa10eOPo=
- python: 3.2
env:
- NUMPYSPEC===1.8.0
- python: 3.3
env:
- NUMPYSPEC===1.8.0
before_install:
- sudo add-apt-repository -y ppa:biometrics/bob
- sudo apt-get update -qq
- sudo apt-get install -qq --force-yes libboost-all-dev libblitz1-dev libhdf5-serial-dev libatlas-dev libatlas-base-dev liblapack-dev
- if [ -n "${NUMPYSPEC}" ]; then sudo apt-get install -qq gfortran; fi
- if [ -n "${NUMPYSPEC}" ]; then pip install --upgrade pip setuptools; fi
- if [ -n "${NUMPYSPEC}" ]; then pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel numpy$NUMPYSPEC ; fi
- pip install --find-links http://wheels.astropy.org/ --find-links http://wheels2.astropy.org/ --use-wheel matplotlib==1.3.0 sphinx nose==1.3.0 coveralls
install:
- python bootstrap.py
- ./bin/buildout
script:
- ./bin/python -c 'from bob.db.mnist import get_config; print(get_config())'
- ./bin/coverage run --source=bob.db.mnist ./bin/nosetests -sv
- ./bin/sphinx-build -b doctest doc sphinx
- ./bin/sphinx-build -b html doc sphinx
after_success:
- coveralls
- ./src/bob.extension/scripts/upload-sphinx.sh
include README.rst bootstrap.py buildout.cfg COPYING
recursive-include docs *.py *.rst
recursive-include doc *.py *.rst
.. vim: set fileencoding=utf-8 :
.. Manuel Guenther <manuel.guenther@idiap.ch>
.. Thu Sep 4 11:35:05 CEST 2014
.. image:: https://travis-ci.org/bioidiap/bob.db.mnist.svg?branch=master
:target: https://travis-ci.org/bioidiap/bob.db.mnist
.. image:: https://coveralls.io/repos/bioidiap/bob.db.mnist/badge.png
:target: https://coveralls.io/r/bioidiap/bob.db.mnist
.. image:: http://img.shields.io/github/tag/bioidiap/bob.db.mnist.png
:target: https://github.com/bioidiap/bob.db.mnist
.. image:: http://img.shields.io/pypi/v/bob.db.mnist.png
:target: https://pypi.python.org/pypi/bob.db.mnist
.. image:: http://img.shields.io/pypi/dm/bob.db.mnist.png
:target: https://pypi.python.org/pypi/bob.db.mnist
================
MNIST Database
================
......@@ -30,10 +45,10 @@ There are a few ways to achieve this:
The package is available in two different distribution formats:
1. You can download it from `PyPI <http://pypi.python.org/pypi/xbob.db.mnist>`_, or
1. You can download it from `PyPI <http://pypi.python.org/pypi/bob.db.mnist>`_, or
2. You can download it in its source form from `its git repository
<https://github.com/bioidiap/xbob.db.mnist>`_.
<https://github.com/bioidiap/bob.db.mnist>`_.
The database raw files must be installed somewhere in your environment.
......@@ -50,11 +65,11 @@ script)::
install_requires=[
...
"xbob.db.mnist",
"bob.db.mnist",
],
Proceed normally with your ``bootstrap/buildout`` steps and you should be all
set. That means you can now import the namespace ``xbob.db.mnist`` into your scripts.
set. That means you can now import the namespace ``bob.db.mnist`` into your scripts.
Modify your buildout.cfg and download from git
==============================================
......@@ -70,10 +85,10 @@ lines::
auto-checkout = *
eggs = bob
...
xbob.db.mnist
bob.db.mnist
[sources]
xbob.db.mnist = git https://github.com/bioidiap/xbob.db.mnist.git
bob.db.mnist = git https://github.com/bioidiap/bob.db.mnist.git
...
......@@ -83,8 +98,8 @@ How to use this database API
After launching the python interpreter (assuming that the environment is properly set up),
you could get the training set as follows::
>>> import xbob.db.mnist
>>> db = xbob.db.mnist.Database('PATH_TO_DATA_FROM_YANN_LECUN_WEBSITE') # 4 binary .gz compressed files
>>> import bob.db.mnist
>>> db = bob.db.mnist.Database('PATH_TO_DATA_FROM_YANN_LECUN_WEBSITE') # 4 binary .gz compressed files
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
In this case, this should return two NumPy arrays:
......@@ -97,17 +112,17 @@ In this case, this should return two NumPy arrays:
If you don't have the data installed on your machine, you can also use the following
set of commands that will:
1. first look for the database in the xbob/db/mnist/ subdirectory and use it if is available
1. first look for the database in the bob/db/mnist/ subdirectory and use it if is available
2. automatically download it from Yann Lecun's website into a temporary folder that will
be erased when the destructor of the xbob.db.mnist database is called.
be erased when the destructor of the bob.db.mnist database is called.
3. automatically download it into the provided directory that will **not** be deleted.
::
>>> import xbob.db.mnist
>>> db = xbob.db.mnist.Database() # Check for the data files locally, and download them if required
>>> import bob.db.mnist
>>> db = bob.db.mnist.Database() # Check for the data files locally, and download them if required
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
>>> del db # delete the temporary downloaded files if any
......@@ -115,7 +130,7 @@ or
::
>>> db = xbob.db.mnist.Database("Directory") # Persistently downloads files into the folder "Directory"
>>> db = bob.db.mnist.Database("Directory") # Persistently downloads files into the folder "Directory"
>>> images, labels = db.data(groups='train', labels=[0,1,2,3,4,5,6,7,8,9])
>>> del db # The download directory stays
......@@ -18,7 +18,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
The MNIST Database is a database of handwritten digits, which has a training
The MNIST Database is a database of handwritten digits, which has a training
set of 60,000 examples, and a test set of 10,000 examples. It is a subset of
a larger set available from NIST. The digits have been size-normalized and
centered in a fixed-size image. You can download the MNIST database from:
......@@ -27,5 +27,21 @@ http://yann.lecun.com/exdb/mnist/
from .query import Database
__all__ = dir()
def get_config():
"""Returns a string containing the configuration information.
"""
import pkg_resources
packages = pkg_resources.require(__name__)
this = packages[0]
deps = packages[1:]
retval = "%s: %s (%s)\n" % (this.key, this.version, this.location)
retval += " - python dependencies:\n"
for d in deps: retval += " - %s: %s (%s)\n" % (d.key, d.version, d.location)
return retval.strip()
# gets sphinx autodoc done right - don't remove it
__all__ = [_ for _ in dir() if not _.startswith('_')]
......@@ -22,7 +22,7 @@
import os
import sys
from bob.db.driver import Interface as BaseInterface
from bob.db.base.driver import Interface as BaseInterface
class Interface(BaseInterface):
......@@ -31,7 +31,7 @@ class Interface(BaseInterface):
def version(self):
import pkg_resources # part of setuptools
return pkg_resources.require('xbob.db.%s' % self.name())[0].version
return pkg_resources.require('bob.db.%s' % self.name())[0].version
def files(self):
return ()
......
......@@ -58,7 +58,7 @@ class Database():
shutil.rmtree(self.m_tmp_dir) # delete directory
except OSError as e:
if e.errno != 2: # code 2 - no such file or directory
raise("xbob.db.mnist: Error while erasing temporarily downloaded data files")
raise("bob.db.mnist: Error while erasing temporarily downloaded data files")
def _db_is_installed(self, directory = None):
from pkg_resources import resource_filename
......
......@@ -20,49 +20,47 @@
"""
import unittest
from . import Database
import bob.db.mnist
class MNISTDatabaseTest(unittest.TestCase):
"""Performs various tests on the MNIST database."""
def test_query():
db = bob.db.mnist.Database()
def test01_query(self):
db = Database()
f = db.labels()
assert len(f) == 10 # number of labels (digits 0 to 9)
for i in range(0,10):
assert i in f
f = db.labels()
self.assertEqual(len(f), 10) # number of labels (digits 0 to 9)
for i in range(0,10):
self.assertTrue(i in f)
f = db.groups()
assert len(f) == 2 # Two groups
assert 'train' in f
assert 'test' in f
f = db.groups()
self.assertEqual(len(f), 2) # Two groups
self.assertTrue('train' in f)
self.assertTrue('test' in f)
# Test the number of samples/labels
d, l = db.data(groups='train')
assert d.shape[0] == 60000
assert d.shape[1] == 784
assert l.shape[0] == 60000
d, l = db.data(groups='test')
assert d.shape[0] == 10000
assert d.shape[1] == 784
assert l.shape[0] == 10000
d, l = db.data()
assert d.shape[0] == 70000
assert d.shape[1] == 784
assert l.shape[0] == 70000
# Test the number of samples/labels
d, l = db.data(groups='train')
self.assertEqual(d.shape[0], 60000)
self.assertEqual(d.shape[1], 784)
self.assertEqual(l.shape[0], 60000)
d, l = db.data(groups='test')
self.assertEqual(d.shape[0], 10000)
self.assertEqual(d.shape[1], 784)
self.assertEqual(l.shape[0], 10000)
d, l = db.data()
self.assertEqual(d.shape[0], 70000)
self.assertEqual(d.shape[1], 784)
self.assertEqual(l.shape[0], 70000)
def test02_download(self):
# tests that the files are downloaded *and stored*, when the directory is specified
import tempfile, os, shutil
temp_dir = tempfile.mkdtemp(prefix='mnist_db_test_')
db = Database(temp_dir)
del db
self.assertTrue(os.path.exists(temp_dir))
def test_download():
# tests that the files are downloaded *and stored*, when the directory is specified
import tempfile, os, shutil
temp_dir = tempfile.mkdtemp(prefix='mnist_db_test_')
db = bob.db.mnist.Database(temp_dir)
del db
assert os.path.exists(temp_dir)
# check that the database works when data is downloaded already
db = Database(temp_dir)
db.data()
del db
# check that the database works when data is downloaded already
db = bob.db.mnist.Database(temp_dir)
assert db.data() is not None
del db
shutil.rmtree(temp_dir)
shutil.rmtree(temp_dir)
; vim: set fileencoding=utf-8 :
; author: Laurent El Shafey <laurent.el-shafey@idiap.ch>
; date: Wed May 8 19:03:45 CEST 2013
; Manuel Guenther <manuel.guenther@idiap.ch>
; Fri Aug 29 14:07:33 CEST 2014
[buildout]
parts = scripts
develop = .
eggs = xbob.db.mnist
eggs = bob.db.mnist
extensions = bob.buildout
mr.developer
auto-checkout = *
develop = src/bob.extension
src/bob.blitz
src/bob.core
src/bob.io.base
src/bob.db.base
.
; options for bob.buildout extension
debug = true
verbose = true
newest = false
[sources]
bob.extension = git https://github.com/bioidiap/bob.extension
bob.blitz = git https://github.com/bioidiap/bob.blitz
bob.core = git https://github.com/bioidiap/bob.core
bob.io.base = git https://github.com/bioidiap/bob.io.base
bob.db.base = git https://github.com/bioidiap/bob.db.base
bob.db.verification.utils = git https://github.com/bioidiap/bob.db.verification.utils
[scripts]
recipe = xbob.buildout:scripts
recipe = bob.buildout:scripts
dependent-scripts=true
......@@ -4,16 +4,16 @@
# Wed May 8 19:05:23 CEST 2013
#
# Copyright (C) 2011-2013 Idiap Research Institute, Martigny, Switzerland
#
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
......@@ -40,7 +40,6 @@ extensions = [
'sphinx.ext.autosummary',
'sphinx.ext.doctest',
'sphinx.ext.intersphinx',
'bob.sphinxext.plot', # ours add source copying to install directory
]
# The viewcode extension appeared only on Sphinx >= 1.0.0
......@@ -77,7 +76,7 @@ copyright = u'%s, Idiap Research Institute' % time.strftime('%Y')
# built documents.
#
# The short X.Y version.
from xbob.db.mnist.driver import Interface
from bob.db.mnist.driver import Interface
version = Interface().version()
# The full version, including alpha/beta/rc tags.
release = version
......@@ -151,7 +150,7 @@ html_favicon = ''
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
#html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
......@@ -248,7 +247,7 @@ man_pages = [
]
# We want to remove all private (i.e. _. or __.__) members
# We want to remove all private (i.e. _. or __.__) members
# that are not in the list of accepted functions
accepted_private_functions = ['__call__']
......@@ -263,10 +262,10 @@ def member_function_test(app, what, name, obj, skip, options):
# test if the method is documented
if not hasattr(obj, '__doc__') or not obj.__doc__:
return True
# Skips selected members in auto-generated documentation. Unfortunately, old
# versions of Boost.Python will not generate a __self__ member for static
# methods and that screws-up Sphinx processing.
# methods and that screws-up Sphinx processing.
if sphinx.__version__ < "1.0":
# We have to remove objects that do not have a __self__ attribute set
import types
......@@ -276,7 +275,7 @@ def member_function_test(app, what, name, obj, skip, options):
return True
return False
# Default processing flags for sphinx
autoclass_content = 'both'
autodoc_member_order = 'bysource'
......
.. vim: set fileencoding=utf-8 :
.. @author: Manuel Guenther <Manuel.Guenther@idiap.ch>
.. @date: Thu Dec 6 12:28:25 CET 2012
==============
User's Guide
==============
.. todo::
Write users guide.
.. vim: set fileencoding=utf-8 :
.. @author: Manuel Guenther <Manuel.Guenther@idiap.ch>
.. @date: Tue Aug 26 09:42:18 CEST 2014
==========================
MNIST Database Protocols
==========================
.. todolist::
Documentation
-------------
.. toctree::
:maxdepth: 2
guide
py_api
Indices and tables
------------------
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
......@@ -2,8 +2,8 @@
.. Laurent El Shafey <Laurent.El-Shafey@idiap.ch>
.. Wed May 8 19:07:55 CEST 2013
================
MNIST Database
================
============
Python API
============
.. automodule:: xbob.db.mnist
.. automodule:: bob.db.mnist
......@@ -22,14 +22,14 @@ from setuptools import setup, find_packages
# parameters that define our package.
setup(
name='xbob.db.mnist',
version='1.0.5a0',
name='bob.db.mnist',
version='2.0.0a0',
description='MNIST Database Access API for Bob',
url='https://pypi.python.org/pypi/xbob.db.mnist',
url='https://pypi.python.org/pypi/bob.db.mnist',
license='GPLv3',
author='Laurent El Shafey',
author_email='laurent.el-shafey@idiap.ch',
keywords='digit recognition, bob, xbob, xbob.db, mnist',
keywords='digit recognition, bob, bob.db, mnist',
long_description=open('README.rst').read(),
# This line is required for any distutils based packaging.
......@@ -39,30 +39,25 @@ setup(
install_requires=[
'setuptools',
'bob', # base signal proc./machine learning library
'bob.db.base',
],
namespace_packages = [
'xbob',
'xbob.db',
],
'bob',
'bob.db',
],
entry_points={
# declare database to bob
'bob.db': [
'mnist = xbob.db.mnist.driver:Interface',
],
# declare tests to bob
'bob.test': [
'mnist = xbob.db.mnist.test:MNISTDatabaseTest',
],
'mnist = bob.db.mnist.driver:Interface',
],
},
},
classifiers = [
'Development Status :: 5 - Production/Stable',
'Development Status :: 4 - Beta',
'Environment :: Console',
'Intended Audience :: Developers',
'Intended Audience :: Education',
......@@ -73,5 +68,5 @@ setup(
'Programming Language :: Python :: 3',
'Topic :: Scientific/Engineering :: Artificial Intelligence',
'Topic :: Database :: Front-Ends',
],
],
)
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment