Skip to content
Snippets Groups Projects
Commit cf31a625 authored by André Anjos's avatar André Anjos :speech_balloon:
Browse files

Rudimentary file reading implemented

parent 2dce6705
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
doc/c_cpp_api.rst
+ 22
− 345
View file @ cf31a625
...@@ -6,11 +6,11 @@ ...@@ -6,11 +6,11 @@
C++ API C++ API
========= =========
The C++ API of ``xbob.core`` allows users to leverage from automatic converters The C++ API of ``xbob.io`` allows users to leverage from automatic converters
for classes in :py:class:`xbob.core.random`. To use the C API, clients should for classes in :py:class:`xbob.io`. To use the C API, clients should first,
first, include the header file ``<xbob.core/random.h>`` on their compilation include the header file ``<xbob.io/api.h>`` on their compilation units and
units and then, make sure to call once ``import_xbob_core_random()`` at their then, make sure to call once ``import_xbob_io()`` at their module
module instantiation, as explained at the `Python manual instantiation, as explained at the `Python manual
<http://docs.python.org/2/extending/extending.html#using-capsules>`_. <http://docs.python.org/2/extending/extending.html#using-capsules>`_.
Here is a dummy C example showing how to include the header and where to call Here is a dummy C example showing how to include the header and where to call
...@@ -18,7 +18,7 @@ the import function: ...@@ -18,7 +18,7 @@ the import function:
.. code-block:: c++ .. code-block:: c++
#include <xbob.core/random.h> #include <xbob.io/api.h>
PyMODINIT_FUNC initclient(void) { PyMODINIT_FUNC initclient(void) {
...@@ -33,363 +33,40 @@ the import function: ...@@ -33,363 +33,40 @@ the import function:
import_blitz_array(); import_blitz_array();
// imports xbob.core.random C-API // imports xbob.core.random C-API
import_xbob_core_random(); import_xbob_io();
} }
.. note:: .. note::
The include directory can be discovered using The include directory can be discovered using
:py:func:`xbob.core.get_include`. :py:func:`xbob.io.get_include`.
Mersenne Twister Random Number Generator (mt19937) Generic Functions
-------------------------------------------------- -----------------
This package contains bindings to ``boost::random::mt19937``, which is a .. cpp:function:: int PyBobIo_AsTypenum(bob::core::array::ElementType et)
powerful random number generator available within the Boost_ C++ library.
.. cpp:type:: PyBoostMt19937Object Converts the input Bob element type into a ``NPY_<TYPE>`` enumeration value.
Returns ``NPY_NOTYPE`` in case of problems, and sets a
:py:class:`RuntimeError`.
The pythonic object representation for a ``boost::random::mt19937`` object. Bob File Support
.. code-block:: c
typedef struct {
PyObject_HEAD
boost::random::mt19937* rng;
} PyBoostMt19937Object;
.. c:member:: boost::random::mt19937* rng
A direct pointer to the boost random number generator. You can use this
pointer in your C/C++ code if required.
.. cpp:function:: int PyBoostMt19937_Check(PyObject* o)
Checks if the input object ``o`` is a ``PyBoostMt19937Object``. Returns
``1`` if it is, and ``0`` otherwise.
.. cpp:function:: int PyBoostMt19937_Converter(PyObject* o, PyBoostMt19937Object** a)
This function is meant to be used with :c:func:`PyArg_ParseTupleAndKeywords`
family of functions in the Python C-API. It checks the input
object to be of type ``PyBoostMt19937Object`` and sets a **new
reference** to it (in ``*a``) if it is the case. Returns ``0`` in case of
failure, ``1`` in case of success.
.. cpp:function:: PyObject* PyBoostMt19937_SimpleNew()
Creates a new instance of :cpp:type:`PyBoostMt19937Object`, with the default
seed. Returns a **new reference**.
.. cpp:function:: PyObject* PyBoostMt19937_NewWithSeed(Py_ssize_t seed)
Creates a new instance of :cpp:type:`PyBoostMt19937Object`, with a user
given seed. Returns a **new reference**.
Distribution API
---------------- ----------------
Together with the boost random number generator ``mt19937``, this package .. cpp:type:: PyBobIoFileObject
provides bindings to these ``boost::random`` distributions:
* Uniform
* Normal (or Gaussian)
* Log-normal
* Gamma
* Binomial
Distributions wrap the random number generator, skewing the distribution of
numbers according to their parametrization. Distributions are *templated*
according to the scalar data types they produce. Different distributions
support a different set of scalar types:
============== =================================================
Distribution Scalars supported
============== =================================================
Uniform bool, int8/16/32/64, uint8/16/32/64, float32/64
Normal float32/64
Log-normal float32/64
Gamma float32/64
Binomial float32/64 (internally using int64)
============== =================================================
.. cpp:type:: PyBoostUniformObject The pythonic object representation for a ``bob::io::File`` object.
The pythonic object representation for a ``boost::random::uniform_*`` .. code-block:: cpp
object.
.. code-block:: c
typedef struct { typedef struct {
PyObject_HEAD PyObject_HEAD
int type_num; boost::shared_ptr<bob::io::File> f;
boost::shared_ptr<void> distro; } PyBobIoFileObject;
} PyUniformObject;
.. c:member:: int type_num;
The NumPy type number of scalars produced by this distribution. Accepted
values match the scalar type produced:
============= ========================================
Scalar type NumPy scalar type number (enumeration)
============= ========================================
bool ``NPY_BOOL``
int8 ``NPY_INT8``
int16 ``NPY_INT16``
int32 ``NPY_INT32``
int64 ``NPY_INT64``
int8 ``NPY_INT8``
int16 ``NPY_INT16``
int32 ``NPY_INT32``
int64 ``NPY_INT64``
float32 ``NPY_FLOAT32``
float64 ``NPY_FLOAT64``
============= ========================================
.. c:member:: boost::shared_ptr<void> distro
A direct pointer to the boost distribution. The underlying allocated type
changes with the scalar that is produced by the distribution:
============= ==============================================
Scalar type C++ data type
============= ==============================================
bool ``boost::random::uniform_smallint<uint8_t>``
int8 ``boost::random::uniform_int<int8_t>``
int16 ``boost::random::uniform_int<int16_t>``
int32 ``boost::random::uniform_int<int32_t>``
int64 ``boost::random::uniform_int<int64_t>``
uint8 ``boost::random::uniform_int<uint8_t>``
uint16 ``boost::random::uniform_int<uint16_t>``
uint32 ``boost::random::uniform_int<uint32_t>``
uint64 ``boost::random::uniform_int<uint64_t>``
float32 ``boost::random::uniform_real<float>``
float64 ``boost::random::uniform_real<double>``
============= ==============================================
In order to use the distribution in your C/C++ code, you must first cast the
shared pointer using ``boost::static_pointer_cast<D>``, with ``D`` matching
one of the distributions listed above, depending on the value of
.. cpp:type:: PyBoostNormalObject
The pythonic object representation for a
``boost::random::normal_distribution`` object.
.. code-block:: c
typedef struct {
PyObject_HEAD
int type_num;
boost::shared_ptr<void> distro;
} PyUniformObject;
.. c:member:: int type_num;
The NumPy type number of scalars produced by this distribution. Accepted
values match the scalar type produced:
============= ========================================
Scalar type NumPy scalar type number (enumeration)
============= ========================================
float32 ``NPY_FLOAT32``
float64 ``NPY_FLOAT64``
============= ========================================
.. c:member:: boost::shared_ptr<void> distro
A direct pointer to the boost distribution. The underlying allocated type
changes with the scalar that is produced by the distribution:
============= ================================================
Scalar type C++ data type
============= ================================================
float32 ``boost::random::normal_distribution<float>``
float64 ``boost::random::normal_distribution<double>``
============= ================================================
.. cpp:type:: PyBoostLogNormalObject
The pythonic object representation for a
``boost::random::lognormal_distribution`` object.
.. code-block:: c
typedef struct {
PyObject_HEAD
int type_num;
boost::shared_ptr<void> distro;
} PyUniformObject;
.. c:member:: int type_num;
The NumPy type number of scalars produced by this distribution. Accepted
values match the scalar type produced:
============= ========================================
Scalar type NumPy scalar type number (enumeration)
============= ========================================
float32 ``NPY_FLOAT32``
float64 ``NPY_FLOAT64``
============= ========================================
.. c:member:: boost::shared_ptr<void> distro
A direct pointer to the boost distribution. The underlying allocated type
changes with the scalar that is produced by the distribution:
============= ===================================================
Scalar type C++ data type
============= ===================================================
float32 ``boost::random::lognormal_distribution<float>``
float64 ``boost::random::lognormal_distribution<double>``
============= ===================================================
.. cpp:type:: PyBoostGammaObject
The pythonic object representation for a
``boost::random::gamma_distribution`` object.
.. code-block:: c
typedef struct {
PyObject_HEAD
int type_num;
boost::shared_ptr<void> distro;
} PyUniformObject;
.. c:member:: int type_num;
The NumPy type number of scalars produced by this distribution. Accepted
values match the scalar type produced:
============= ========================================
Scalar type NumPy scalar type number (enumeration)
============= ========================================
float32 ``NPY_FLOAT32``
float64 ``NPY_FLOAT64``
============= ========================================
.. c:member:: boost::shared_ptr<void> distro
A direct pointer to the boost distribution. The underlying allocated type
changes with the scalar that is produced by the distribution:
============= ===============================================
Scalar type C++ data type
============= ===============================================
float32 ``boost::random::gamma_distribution<float>``
float64 ``boost::random::gamma_distribution<double>``
============= ===============================================
.. cpp:type:: PyBoostBinomialObject
The pythonic object representation for a
``boost::random::binomial_distribution`` object.
.. code-block:: c
typedef struct {
PyObject_HEAD
int type_num;
boost::shared_ptr<void> distro;
} PyUniformObject;
.. c:member:: int type_num;
The NumPy type number of scalars produced by this distribution. Accepted
values match the scalar type produced:
============= ========================================
Scalar type NumPy scalar type number (enumeration)
============= ========================================
float32 ``NPY_FLOAT32``
float64 ``NPY_FLOAT64``
============= ========================================
.. c:member:: boost::shared_ptr<void> distro
A direct pointer to the boost distribution. The underlying allocated type
changes with the scalar that is produced by the distribution:
============= ==========================================================
Scalar type C++ data type
============= ==========================================================
float32 ``boost::random::binomial_distribution<int64_t,float>``
float64 ``boost::random::binomial_distribution<int64_t,double>``
============= ==========================================================
.. cpp:function:: int PyBoostUniform_Check(PyObject* o)
.. cpp:function:: int PyBoostNormal_Check(PyObject* o)
.. cpp:function:: int PyBoostLogNormal_Check(PyObject* o)
.. cpp:function:: int PyBoostGamma_Check(PyObject* o)
.. cpp:function:: int PyBoostBinomial_Check(PyObject* o)
Checks if the input object ``o`` is a ``PyBoost<Distribution>Object``.
Returns ``1`` if it is, and ``0`` otherwise.
.. cpp:function:: int PyBoostUniform_Converter(PyObject* o, PyBoostUniformObject** a)
.. cpp:function:: int PyBoostNormal_Converter(PyObject* o, PyBoostNormalObject** a)
.. cpp:function:: int PyBoostLogNormal_Converter(PyObject* o, PyBoostLogNormalObject** a)
.. cpp:function:: int PyBoostGamma_Converter(PyObject* o, PyBoostGammaObject** a)
.. cpp:function:: int PyBoostBinomial_Converter(PyObject* o, PyBoostBinomialObject** a)
This function is meant to be used with :c:func:`PyArg_ParseTupleAndKeywords`
family of functions in the Python C-API. It checks the input object to be of
type ``PyBoost<Distribution>Object`` and returns a **new reference** to it
(in ``*a``) if it is the case. Returns ``0`` in case of failure, ``1`` in
case of success.
.. cpp:function:: PyObject* PyBoostUniform_SimpleNew(int type_num, PyObject* min, PyObject* max)
Creates a new instance of :cpp:type:`PyBoostUniformObject`, with the input
scalar establishing the minimum and the maximum of the distribution. Note
that ``bool`` distributions will raise an exception if one tries to set the
minimum and the maximum, since that is non-sensical.
The parameter ``type_num`` may be set to one of the supported ``NPY_``
enumeration values (e.g. ``NPY_UINT16``).
.. warning::
For integral uniform distributions the range of numbers produced is
defined as :math:`[min, max]`. For real-valued distributions, the range of
numbers produced lies on the interval :math:`[min, max[`.
.. cpp:function:: PyObject* PyBoostNormal_SimpleNew(int type_num, PyObject* mean, PyObject* sigma)
.. cpp:function:: PyObject* PyBoostLogNormal_SimpleNew(int type_num, PyObject* mean, PyObject* sigma)
.. cpp:function:: PyObject* PyBoostGamma_SimpleNew(int type_num, PyObject* alpha, PyObject* beta)
.. cpp:function:: PyObject* PyBoostBinomial_SimpleNew(int type_num, PyObject* t, PyObject* p)
Depending on the distribution, which may be one of ``Normal``,
``LogNormal``, ``Gamma`` or ``Binomial``, each of the parameters assume a
different function:
============== ============= ============================ .. cpp:member:: boost::shared_ptr<bob::io::File> f
Distribution Parameter 1 Parameter 2
============== ============= ============================
Normal mean sigma (standard deviation)
LogNormal mean sigma (standard deviation)
Gamma alpha beta
Binomial t p
============== ============= ============================
The parameter ``type_num`` may be set to one of the supported ``NPY_`` A pointer to a file being read or written.
enumeration values (e.g. ``NPY_FLOAT64``).
.. include:: links.rst .. include:: links.rst
doc/py_api.rst
+ 7
− 136
View file @ cf31a625
...@@ -2,152 +2,23 @@ ...@@ -2,152 +2,23 @@
.. Andre Anjos <andre.dos.anjos@gmail.com> .. Andre Anjos <andre.dos.anjos@gmail.com>
.. Tue 15 Oct 17:41:52 2013 .. Tue 15 Oct 17:41:52 2013
.. testsetup:: coretest .. testsetup:: iotest
import numpy import numpy
import xbob.core import xbob.io
============ ============
User Guide User Guide
============ ============
Array Conversion Basic I/O
---------------- ---------
The function :py:func:`xbob.core.convert` allows you to convert objects of type
:py:class:`numpy.ndarray` between different types, with range compression or
decompression. For example, here we demonstrate a conversion using default
ranges. In this type of conversion, our implementation will assume that the
source array contains values within the range of ``uint8_t`` numbers and will
expand it to the range of ``uint16_t`` numbers, as desired by the programmer:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> x = numpy.array([0,255,0,255,0,255], 'uint8').reshape(2,3)
>>> x
array([[ 0, 255, 0],
[255, 0, 255]], dtype=uint8)
>>> xbob.core.convert(x, 'uint16')
array([[ 0, 65535, 0],
[65535, 0, 65535]], dtype=uint16)
The user can optionally specify source, destination ranges or both. For
example:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> x = numpy.array([0, 10, 20, 30, 40], 'uint8')
>>> xbob.core.convert(x, 'float64', source_range=(0,40), dest_range=(0.,1.))
array([ 0. , 0.25, 0.5 , 0.75, 1. ])
Any range not specified is assumed to default on the type range.
Random Number Generation
------------------------
You can build a new random number generator (RNG) of type
:py:class:`xbob.core.random.mt19937` using one of two possible ways:
1. Use the default constructor, which initializes with the default seed:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> xbob.core.random.mt19937()
xbob.core.random.mt19937()
2. Pass a seed while initializing:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> rng = xbob.core.random.mt19937(34)
RNGs can be compared for equality. The ``==`` operator checks if both
generators are on the exact same state and would generate the same sequence of
numbers when exposed to the same distributions. For example:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> rng1 = xbob.core.random.mt19937(111)
>>> rng2 = xbob.core.random.mt19937(111)
>>> rng1 == rng2
True
>>> rng3 = xbob.core.random.mt19937(12)
>>> rng1 == rng3
False
The seed can be re-initialized at any point in time, which can be used to sync
two RNGs:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> rng3.seed(111)
>>> rng1 == rng3
True
Distributions skew numbers produced by the RNG so they look like the
parameterized distribution. By calling a distribution with an RNG, one
effectively generates random numbers:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> rng = xbob.core.random.mt19937()
>>> # creates an uniform distribution of integers inside [0, 10]
>>> u = xbob.core.random.uniform(int, 0, 10)
>>> u(rng) # doctest: +SKIP
8
At our reference guide (see below), you will find more implemented
distributions you can use on your programs. To simplify the task of generating
random numbers, we provide a class that mimics the behavior of
``boost::random::variate_generator``, in Python:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> ugen = xbob.core.random.variate_generator(rng, u)
>>> ugen() # doctest: +SKIP
6
You can also pass an optional shape when you call the variate generator, in
which case it generates a :py:class:`numpy.ndarray` of the specified size:
.. doctest:: coretest
:options: +NORMALIZE_WHITESPACE
>>> ugen((3,3)) # doctest: +SKIP
array([[ 3, 1, 6],
[ 3, 2, 6],
[10, 10, 10]])
Reference Reference
--------- ---------
This section includes information for using the pure Python API of This section includes information for using the pure Python API of ``xbob.io``.
``xbob.core``.
.. autofunction:: xbob.core.get_include
.. autofunction:: xbob.core.convert
.. autoclass:: xbob.core.random.mt19937
.. autoclass:: xbob.core.random.uniform
.. autoclass:: xbob.core.random.normal
.. autoclass:: xbob.core.random.lognormal
.. autoclass:: xbob.core.random.gamma
.. autoclass:: xbob.core.random.binomial .. autofunction:: xbob.io.get_include
.. autoclass:: xbob.core.random.variate_generator .. autoclass:: xbob.io.file
setup.py
+ 3
− 1
View file @ cf31a625
...@@ -62,8 +62,9 @@ else: ...@@ -62,8 +62,9 @@ else:
# Local include directory # Local include directory
import os import os
package_dir = os.path.dirname(os.path.realpath(__file__)) package_dir = os.path.dirname(os.path.realpath(__file__))
package_base = os.path.join(package_dir, 'xbob', 'io')
package_dir = os.path.join(package_dir, 'xbob', 'io', 'include') package_dir = os.path.join(package_dir, 'xbob', 'io', 'include')
include_dirs = [package_dir] include_dirs = [package_base, package_dir]
# Define package version # Define package version
version = '2.0.0a0' version = '2.0.0a0'
...@@ -98,6 +99,7 @@ setup( ...@@ -98,6 +99,7 @@ setup(
ext_modules = [ ext_modules = [
Extension("xbob.io._library", Extension("xbob.io._library",
[ [
"xbob/io/bobskin.cpp",
"xbob/io/file.cpp", "xbob/io/file.cpp",
"xbob/io/main.cpp", "xbob/io/main.cpp",
], ],
......
xbob/io/__init__.py
+ 1
− 1
View file @ cf31a625
from ._library import __version__, __api_version__ from ._library import __version__, __api_version__, file
def get_include(): def get_include():
"""Returns the directory containing the C/C++ API include directives""" """Returns the directory containing the C/C++ API include directives"""
......
xbob/io/bobskin.cpp 0 → 100644
+ 50
− 0
View file @ cf31a625
/**
* @author Andre Anjos <andre.anjos@idiap.ch>
* @date Wed 6 Nov 07:57:57 2013
*
* @brief Implementation of our bobskin class
*/
#include <numpy/arrayobject.h>
#include <stdexcept>
bobskin::bobskin(PyObject* array, bob::core::array::ElementType& eltype) {
if (!PyArray_CheckExact(array)) {
PyErr_SetString(PyExc_TypeError, "input object to bobskin constructor is not a numpy.ndarray");
throw std::runtime_error();
}
m_type.set(eltype, PyArray_NDIM(array), PyArray_DIMS(array),
PyArray_STRIDES(array));
m_ptr = PyArray_DATA(array);
}
bobskin::~bobskin() { }
void bobskin::set(const interface&) {
PyErr_SetString(PyExc_NotImplementedError, "setting C++ bobskin with (const interface&) is not implemented - DEBUG ME!");
throw std::runtime_error();
}
void bobskin::set(boost::shared_ptr<interface> other);
PyErr_SetString(PyExc_NotImplementedError, "setting C++ bobskin with (boost::shared_ptr<interface>) is not implemented - DEBUG ME!");
throw std::runtime_error();
}
void bobskin::set (const bob::core::array::typeinfo& req) {
PyErr_SetString(PyExc_NotImplementedError, "setting C++ bobskin with (const typeinfo&) implemented - DEBUG ME!");
throw std::runtime_error();
}
boost::shared_ptr<void> bobskin::owner() {
PyErr_SetString(PyExc_NotImplementedError, "acquiring non-const owner from C++ bobskin is not implemented - DEBUG ME!");
throw std::runtime_error();
}
boost::shared_ptr<const void> bobskin::owner() const {
PyErr_SetString(PyExc_NotImplementedError, "acquiring const owner from C++ bobskin is not implemented - DEBUG ME!");
throw std::runtime_error();
}
xbob/io/bobskin.h 0 → 100644
+ 72
− 0
View file @ cf31a625
/**
* @author Andre Anjos <andre.anjos@idiap.ch>
* @date Tue 5 Nov 22:09:07 2013
*
* @brief A pythonic version of bob::core::array::interface, with minimal
* functionality.
*/
#include <Python.h>
#include <bob/core/array.h>
/**
* Wraps a PyArrayObject such that we can access it from bob::io
*/
class bobskin: public bob::core::array::interface {
public: //api
/**
* @brief Builds a new array an array like object
*/
bobskin(PyObject* array, bob::core::array::ElementType eltype);
/**
* @brief By default, the interface is never freed. You must override
* this method to do something special for your class type.
*/
virtual ~bobskin();
/**
* @brief Copies the data from another interface.
*/
virtual void set(const interface& other);
/**
* @brief Refers to the data of another interface.
*/
virtual void set(boost::shared_ptr<interface> other);
/**
* @brief Re-allocates this interface taking into consideration new
* requirements. The internal memory should be considered uninitialized.
*/
virtual void set (const bob::core::array::typeinfo& req);
/**
* @brief Type information for this interface.
*/
virtual const bob::core::array::typeinfo& type() const { return m_type; }
/**
* @brief Borrows a reference from the underlying memory. This means
* this object continues to be responsible for deleting the memory and
* you should make sure that it outlives the usage of the returned
* pointer.
*/
virtual void* ptr() { return m_ptr; }
virtual const void* ptr() const { return m_ptr; }
/**
* @brief Returns a representation of the internal cache using shared
* pointers.
*/
virtual boost::shared_ptr<void> owner();
virtual boost::shared_ptr<const void> owner() const;
private: //representation
bob::core::array::typeinfo m_type; ///< type information
void* m_ptr; ///< pointer to the data
};
xbob/io/file.cpp
+ 164
− 18
View file @ cf31a625
...@@ -9,8 +9,11 @@ ...@@ -9,8 +9,11 @@
#include <xbob.io/api.h> #include <xbob.io/api.h>
#include <bob/io/CodecRegistry.h> #include <bob/io/CodecRegistry.h>
#include <bob/io/utils.h> #include <bob/io/utils.h>
#include <numpy/arrayobject.h>
#include <stdexcept> #include <stdexcept>
#include <bobskin.h>
#define FILETYPE_NAME file #define FILETYPE_NAME file
PyDoc_STRVAR(s_file_str, BOOST_PP_STRINGIZE(XBOB_IO_MODULE_PREFIX) "." BOOST_PP_STRINGIZE(FILETYPE_NAME)); PyDoc_STRVAR(s_file_str, BOOST_PP_STRINGIZE(XBOB_IO_MODULE_PREFIX) "." BOOST_PP_STRINGIZE(FILETYPE_NAME));
...@@ -99,9 +102,161 @@ pretend_extension\n\ ...@@ -99,9 +102,161 @@ pretend_extension\n\
); );
static PyObject* PyBobIoFile_Repr(PyBobIoFileObject* self) { static PyObject* PyBobIoFile_Repr(PyBobIoFileObject* self) {
return PyUnicode_FromFormat("%s()", s_file_str); return
# if PY_VERSION_HEX >= 0x03000000
PyUnicode_FromFormat
# else
PyString_FromFormat
# endif
("%s(filename='%s', codec='%s')", s_file_str, self->f->filename().c_str(),
self->f->name().c_str());
}
static PyObject* PyBobIoFile_Filename(PyBobIoFileObject* self) {
return
# if PY_VERSION_HEX >= 0x03000000
PyUnicode_FromString
# else
PyString_FromString
# endif
(self->f->filename().c_str());
}
static PyObject* PyBobIoFile_CodecName(PyBobIoFileObject* self) {
return
# if PY_VERSION_HEX >= 0x03000000
PyUnicode_FromString
# else
PyString_FromString
# endif
(self->f->name().c_str());
}
PyDoc_STRVAR(s_filename_str, "filename");
PyDoc_STRVAR(s_filename_doc,
"The path to the file being read/written"
);
PyDoc_STRVAR(s_codec_name_str, "codec_name");
PyDoc_STRVAR(s_codec_name_doc,
"Name of the File class implementation -- available for\n\
compatibility reasons with the previous versions of this\n\
library."
);
static PyGetSetDef PyBobIoFile_getseters[] = {
{
s_filename_str,
(getter)PyBobIoFile_Filename,
0,
s_filename_doc,
0,
},
{
s_codec_name_str,
(getter)PyBobIoFile_CodecName,
0,
s_codec_name_doc,
0,
},
{0} /* Sentinel */
};
static Py_ssize_t PyBobIoFile_Len (PyBobIoFileObject* self) {
Py_ssize_t retval = self->f->size();
return retval;
}
int PyBobIo_AsTypenum (bob::core::array::ElementType type) {
switch(type) {
case bob::core::array::t_bool:
return NPY_BOOL;
case bob::core::array::t_int8:
return NPY_INT8;
case bob::core::array::t_int16:
return NPY_INT16;
case bob::core::array::t_int32:
return NPY_INT32;
case bob::core::array::t_int64:
return NPY_INT64;
case bob::core::array::t_uint8:
return NPY_UINT8;
case bob::core::array::t_uint16:
return NPY_UINT16;
case bob::core::array::t_uint32:
return NPY_UINT32;
case bob::core::array::t_uint64:
return NPY_UINT64;
case bob::core::array::t_float32:
return NPY_FLOAT32;
case bob::core::array::t_float64:
return NPY_FLOAT64;
#ifdef NPY_FLOAT128
case bob::core::array::t_float128:
return NPY_FLOAT128;
#endif
case bob::core::array::t_complex64:
return NPY_COMPLEX64;
case bob::core::array::t_complex128:
return NPY_COMPLEX128;
#ifdef NPY_COMPLEX256
case bob::core::array::t_complex256:
return NPY_COMPLEX256;
#endif
default:
PyErr_Format(PyExc_TypeError, "unsupported Bob/C++ element type (%s)", bob::core::array::stringize(type));
}
return NPY_NOTYPE;
} }
static PyObject* PyBobIoFile_GetItem (PyBobIoFileObject* self, Py_ssize_t i) {
if (i < 0 || i >= self->f->size()) {
PyErr_SetString(PyExc_IndexError, "file index out of range");
return 0;
}
const bob::core::array::typeinfo& info = self->f->type();
npy_intp shape[NPY_MAXDIMS];
for (i=0; i<info.nd; ++i) shape[i] = info.shape[i];
int type_num = PyBobIo_AsTypenum(info.dtype);
if (type_num == NPY_NOTYPE) return 0; ///< failure
PyObject* retval = PyArray_SimpleNew(info.nd, shape, type_num);
bobskin skin(retval, info.dtype);
try {
self->f->read(skin, i);
}
catch (std::runtime_error& e) {
return 0;
}
catch (std::exception& e) {
PyErr_Format(PyExc_RuntimeError, "caught std::exception while reading file `%s': %s", self->f->filename().c_str(), e.what());
return 0;
}
catch (...) {
PyErr_Format(PyExc_RuntimeError, "caught unknown while reading file `%s'", self->f->filename().c_str());
return 0;
}
return retval;
}
static PySequenceMethods PyBobIoFile_Sequence = {
(lenfunc)PyBobIoFile_Len,
0, /* concat */
0, /* repeat */
(ssizeargfunc)PyBobIoFile_GetItem,
0 /* slice */
};
PyTypeObject PyBobIoFile_Type = { PyTypeObject PyBobIoFile_Type = {
PyObject_HEAD_INIT(0) PyObject_HEAD_INIT(0)
0, /*ob_size*/ 0, /*ob_size*/
...@@ -115,7 +270,7 @@ PyTypeObject PyBobIoFile_Type = { ...@@ -115,7 +270,7 @@ PyTypeObject PyBobIoFile_Type = {
0, /*tp_compare*/ 0, /*tp_compare*/
(reprfunc)PyBobIoFile_Repr, /*tp_repr*/ (reprfunc)PyBobIoFile_Repr, /*tp_repr*/
0, /*tp_as_number*/ 0, /*tp_as_number*/
0, /*tp_as_sequence*/ &PyBobIoFile_Sequence, /*tp_as_sequence*/
0, /*tp_as_mapping*/ 0, /*tp_as_mapping*/
0, /*tp_hash */ 0, /*tp_hash */
0, /*tp_call*/ 0, /*tp_call*/
...@@ -133,7 +288,7 @@ PyTypeObject PyBobIoFile_Type = { ...@@ -133,7 +288,7 @@ PyTypeObject PyBobIoFile_Type = {
0, /* tp_iternext */ 0, /* tp_iternext */
0, //PyBobIoFile_methods, /* tp_methods */ 0, //PyBobIoFile_methods, /* tp_methods */
0, /* tp_members */ 0, /* tp_members */
0, /* tp_getset */ PyBobIoFile_getseters, /* tp_getset */
0, /* tp_base */ 0, /* tp_base */
0, /* tp_dict */ 0, /* tp_dict */
0, /* tp_descr_get */ 0, /* tp_descr_get */
...@@ -157,16 +312,6 @@ static object file_read(bob::io::File& f, size_t index) { ...@@ -157,16 +312,6 @@ static object file_read(bob::io::File& f, size_t index) {
return a.pyobject(); //shallow copy return a.pyobject(); //shallow copy
} }
static boost::shared_ptr<bob::io::File> string_open1 (const std::string& filename,
const std::string& mode) {
return bob::io::open(filename, mode[0]);
}
static boost::shared_ptr<bob::io::File> string_open2 (const std::string& filename,
const std::string& mode, const std::string& pretend_extension) {
return bob::io::open(filename, mode[0], pretend_extension);
}
static void file_write(bob::io::File& f, object array) { static void file_write(bob::io::File& f, object array) {
bob::python::py_array a(array, object()); bob::python::py_array a(array, object());
f.write(a); f.write(a);
...@@ -189,18 +334,19 @@ static dict extensions() { ...@@ -189,18 +334,19 @@ static dict extensions() {
void bind_io_file() { void bind_io_file() {
class_<bob::io::File, boost::shared_ptr<bob::io::File>, boost::noncopyable>("File", "Abstract base class for all Array/Arrayset i/o operations", no_init)
.def("__init__", make_constructor(string_open1, default_call_policies(), (arg("filename"), arg("mode"))), "Opens a (supported) file for reading arrays. The mode is a **single** character which takes one of the following values: 'r' - opens the file for read-only operations; 'w' - truncates the file and open it for reading and writing; 'a' - opens the file for reading and writing w/o truncating it.")
.def("__init__", make_constructor(string_open2, default_call_policies(), (arg("filename"), arg("mode"), arg("pretend_extension"))), "Opens a (supported) file for reading arrays but pretends its extension is as given by the last parameter - this way you can, potentially, override the default encoder/decoder used to read and write on the file. The mode is a **single** character which takes one of the following values: 'r' - opens the file for read-only operations; 'w' - truncates the file and open it for reading and writing; 'a' - opens the file for reading and writing w/o truncating it.")
.add_property("filename", make_function(&bob::io::File::filename, return_value_policy<copy_const_reference>()), "The path to the file being read/written")
.add_property("type_all", make_function(&bob::io::File::type_all, return_value_policy<copy_const_reference>()), "Typing information to load all of the file at once") .add_property("type_all", make_function(&bob::io::File::type_all, return_value_policy<copy_const_reference>()), "Typing information to load all of the file at once")
.add_property("type", make_function(&bob::io::File::type, return_value_policy<copy_const_reference>()), "Typing information to load the file as an Arrayset") .add_property("type", make_function(&bob::io::File::type, return_value_policy<copy_const_reference>()), "Typing information to load the file as an Arrayset")
.add_property("codec_name", make_function(&bob::io::File::name, return_value_policy<copy_const_reference>()), "Name of the File class implementation -- for compatibility reasons with the previous versions of this library")
.def("read", &file_read_all, (arg("self")), "Reads the whole contents of the file into a NumPy ndarray") .def("read", &file_read_all, (arg("self")), "Reads the whole contents of the file into a NumPy ndarray")
.def("write", &file_write, (arg("self"), arg("array")), "Writes an array into the file, truncating it first") .def("write", &file_write, (arg("self"), arg("array")), "Writes an array into the file, truncating it first")
.def("__len__", &bob::io::File::size, (arg("self")), "Size of the file if it is supposed to be read as a set of arrays instead of performing a single read") .def("__len__", &bob::io::File::size, (arg("self")), "Size of the file if it is supposed to be read as a set of arrays instead of performing a single read")
.def("read", &file_read, (arg("self"), arg("index")), "Reads a single array from the file considering it to be an arrayset list") .def("read", &file_read, (arg("self"), arg("index")), "Reads a single array from the file considering it to be an arrayset list")
.def("__getitem__", &file_read, (arg("self"), arg("index")), "Reads a single array from the file considering it to be an arrayset list") .def("__getitem__", &file_read, (arg("self"), arg("index")), "Reads a single array from the file considering it to be an arrayset list")
.def("append", &file_append, (arg("self"), arg("array")), "Appends an array to a file. Compatibility requirements may be enforced.") .def("append", &file_append, (arg("self"), arg("array")), "Appends an array to a file. Compatibility requirements may be enforced.")
; ;
......
xbob/io/include/xbob.io/api.h
+ 21
− 1
View file @ cf31a625
...@@ -44,8 +44,16 @@ typedef struct { ...@@ -44,8 +44,16 @@ typedef struct {
#define PyBobIoFile_Type_NUM 1 #define PyBobIoFile_Type_NUM 1
#define PyBobIoFile_Type_TYPE PyTypeObject #define PyBobIoFile_Type_TYPE PyTypeObject
/************************
* I/O generic bindings *
************************/
#define PyBobIo_AsTypenum_NUM 2
#define PyBobIo_AsTypenum_RET int
#define PyBobIo_AsTypenum_PROTO (bob::core::array::ElementType et)
/* Total number of C API pointers */ /* Total number of C API pointers */
#define PyXbobIo_API_pointers 2 #define PyXbobIo_API_pointers 3
#ifdef XBOB_IO_MODULE #ifdef XBOB_IO_MODULE
...@@ -63,6 +71,12 @@ typedef struct { ...@@ -63,6 +71,12 @@ typedef struct {
extern PyBobIoFile_Type_TYPE PyBobIoFile_Type; extern PyBobIoFile_Type_TYPE PyBobIoFile_Type;
/************************
* I/O generic bindings *
************************/
PyBobIo_AsTypenum_RET PyBobIo_AsTypenum PyBobIo_AsTypenum_PROTO;
#else #else
/* This section is used in modules that use `blitz.array's' C-API */ /* This section is used in modules that use `blitz.array's' C-API */
...@@ -103,6 +117,12 @@ typedef struct { ...@@ -103,6 +117,12 @@ typedef struct {
# define PyBobIoFile_Type (*(PyBobIoFile_Type_TYPE *)PyXbobIo_API[PyBobIoFile_Type_NUM]) # define PyBobIoFile_Type (*(PyBobIoFile_Type_TYPE *)PyXbobIo_API[PyBobIoFile_Type_NUM])
/************************
* I/O generic bindings *
************************/
# define PyBobIo_AsTypenum (*(PyBobIo_AsTypenum_RET (*)PyBobIo_AsTypenum_PROTO) PyBlitzArray_API[PyBobIo_AsTypenum_NUM])
/** /**
* Returns -1 on error, 0 on success. PyCapsule_Import will set an exception * Returns -1 on error, 0 on success. PyCapsule_Import will set an exception
* if there's an error. * if there's an error.
......
xbob/io/main.cpp
+ 11
− 0
View file @ cf31a625
...@@ -43,6 +43,11 @@ PyMODINIT_FUNC ENTRY_FUNCTION(XBOB_IO_MODULE_NAME) (void) { ...@@ -43,6 +43,11 @@ PyMODINIT_FUNC ENTRY_FUNCTION(XBOB_IO_MODULE_NAME) (void) {
static void* PyXbobIo_API[PyXbobIo_API_pointers]; static void* PyXbobIo_API[PyXbobIo_API_pointers];
/* exhaustive list of C APIs */ /* exhaustive list of C APIs */
/**************
* Versioning *
**************/
PyXbobIo_API[PyXbobIo_APIVersion_NUM] = (void *)&PyXbobIo_APIVersion; PyXbobIo_API[PyXbobIo_APIVersion_NUM] = (void *)&PyXbobIo_APIVersion;
/***************************** /*****************************
...@@ -51,6 +56,12 @@ PyMODINIT_FUNC ENTRY_FUNCTION(XBOB_IO_MODULE_NAME) (void) { ...@@ -51,6 +56,12 @@ PyMODINIT_FUNC ENTRY_FUNCTION(XBOB_IO_MODULE_NAME) (void) {
PyXbobIo_API[PyBobIoFile_Type_NUM] = (void *)&PyBobIoFile_Type; PyXbobIo_API[PyBobIoFile_Type_NUM] = (void *)&PyBobIoFile_Type;
/************************
* I/O generic bindings *
************************/
PyBlitzArray_API[PyBobIo_AsTypenum_NUM] = (void *)PyBobIo_AsTypenum;
/* imports the NumPy C-API */ /* imports the NumPy C-API */
import_array(); import_array();
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment