Skip to content

Commit 9975c39

Browse files
pitroupitrou
committed
Issue #2396: backport the memoryview object.
1 parent dfeb3ab commit 9975c39

15 files changed

Lines changed: 1672 additions & 26 deletions

File tree

‎Doc/c-api/buffer.rst‎

Lines changed: 291 additions & 3 deletions
Large diffs are not rendered by default.

‎Doc/c-api/objbuffer.rst‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,15 @@
22

33
.. _abstract-buffer:
44

5-
Buffer Protocol
6-
===============
5+
6+
Old Buffer Protocol
7+
===================
8+
9+
This section describes the legacy buffer protocol, which has been introduced
10+
in Python 1.6. It is still supported but deprecated in the Python 2.x series.
11+
Python 3.0 introduces a new buffer protocol which fixes weaknesses and
12+
shortcomings of the protocol, and has been backported to Python 2.6.
13+
See :ref:`bufferobjects` for more information.
714

815

916
.. cfunction:: int PyObject_AsCharBuffer(PyObject *obj, const char **buffer, Py_ssize_t *buffer_len)

‎Doc/library/functions.rst‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -680,6 +680,13 @@ available. They are listed here in alphabetical order.
680680
Added support for the optional *key* argument.
681681

682682

683+
.. function:: memoryview(obj)
684+
:noindex:
685+
686+
Return a "memory view" object created from the given argument. See
687+
:ref:`typememoryview` for more information.
688+
689+
683690
.. function:: min(iterable[, args...][key])
684691

685692
With a single argument *iterable*, return the smallest item of a non-empty

‎Doc/library/stdtypes.rst‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2354,6 +2354,104 @@ the particular object.
23542354
state.
23552355

23562356

2357+
.. _typememoryview:
2358+
2359+
memoryview Types
2360+
================
2361+
2362+
:class:`memoryview`\s allow Python code to access the internal data of an object
2363+
that supports the buffer protocol without copying. Memory can be interpreted as
2364+
simple bytes or complex data structures.
2365+
2366+
.. class:: memoryview(obj)
2367+
2368+
Create a :class:`memoryview` that references *obj*. *obj* must support the
2369+
buffer protocol. Builtin objects that support the buffer protocol include
2370+
:class:`str` and :class:`bytearray` (but not :class:`unicode`).
2371+
2372+
``len(view)`` returns the total number of bytes in the memoryview, *view*.
2373+
2374+
A :class:`memoryview` supports slicing to expose its data. Taking a single
2375+
index will return a single byte. Full slicing will result in a subview::
2376+
2377+
>>> v = memoryview('abcefg')
2378+
>>> v[1]
2379+
'b'
2380+
>>> v[-1]
2381+
'g'
2382+
>>> v[1:4]
2383+
<memory at 0x77ab28>
2384+
>>> str(v[1:4])
2385+
'bce'
2386+
>>> v[3:-1]
2387+
<memory at 0x744f18>
2388+
>>> str(v[4:-1])
2389+
'f'
2390+
2391+
If the object the memory view is over supports changing its data, the
2392+
memoryview supports slice assignment::
2393+
2394+
>>> data = bytearray('abcefg')
2395+
>>> v = memoryview(data)
2396+
>>> v.readonly
2397+
False
2398+
>>> v[0] = 'z'
2399+
>>> data
2400+
bytearray(b'zbcefg')
2401+
>>> v[1:4] = '123'
2402+
>>> data
2403+
bytearray(b'z123fg')
2404+
>>> v[2] = 'spam'
2405+
Traceback (most recent call last):
2406+
File "<stdin>", line 1, in <module>
2407+
ValueError: cannot modify size of memoryview object
2408+
2409+
Notice how the size of the memoryview object can not be changed.
2410+
2411+
2412+
:class:`memoryview` has two methods:
2413+
2414+
.. method:: tobytes()
2415+
2416+
Return the data in the buffer as a bytestring (an object of class
2417+
:class:`str`).
2418+
2419+
.. method:: tolist()
2420+
2421+
Return the data in the buffer as a list of integers. ::
2422+
2423+
>>> memoryview(b'abc').tolist()
2424+
[97, 98, 99]
2425+
2426+
There are also several readonly attributes available:
2427+
2428+
.. attribute:: format
2429+
2430+
A string containing the format (in :mod:`struct` module style) for each
2431+
element in the view. This defaults to ``'B'``, a simple bytestring.
2432+
2433+
.. attribute:: itemsize
2434+
2435+
The size in bytes of each element of the memoryview.
2436+
2437+
.. attribute:: shape
2438+
2439+
A tuple of integers the length of :attr:`ndim` giving the shape of the
2440+
memory as a N-dimensional array.
2441+
2442+
.. attribute:: ndim
2443+
2444+
An integer indicating how many dimensions of a multi-dimensional array the
2445+
memory represents.
2446+
2447+
.. attribute:: strides
2448+
2449+
A tuple of integers the length of :attr:`ndim` giving the size in bytes to
2450+
access each element for each dimension of the array.
2451+
2452+
.. memoryview.suboffsets isn't documented because it only seems useful for C
2453+
2454+
23572455
.. _typecontextmanager:
23582456

23592457
Context Manager Types

‎Doc/tutorial/modules.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -328,8 +328,8 @@ want a list of those, they are defined in the standard module
328328
'enumerate', 'eval', 'execfile', 'exit', 'file', 'filter', 'float',
329329
'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex',
330330
'id', 'input', 'int', 'intern', 'isinstance', 'issubclass', 'iter',
331-
'len', 'license', 'list', 'locals', 'long', 'map', 'max', 'min',
332-
'object', 'oct', 'open', 'ord', 'pow', 'property', 'quit', 'range',
331+
'len', 'license', 'list', 'locals', 'long', 'map', 'max', 'memoryview',
332+
'min', 'object', 'oct', 'open', 'ord', 'pow', 'property', 'quit', 'range',
333333
'raw_input', 'reduce', 'reload', 'repr', 'reversed', 'round', 'set',
334334
'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super',
335335
'tuple', 'type', 'unichr', 'unicode', 'vars', 'xrange', 'zip']

‎Include/Python.h‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@
9292
#endif
9393
#include "rangeobject.h"
9494
#include "stringobject.h"
95-
/* #include "memoryobject.h" */
95+
#include "memoryobject.h"
9696
#include "bufferobject.h"
9797
#include "bytesobject.h"
9898
#include "bytearrayobject.h"

‎Include/memoryobject.h‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
/* Memory view object. In Python this is available as "memoryview". */
2+
3+
#ifndef Py_MEMORYOBJECT_H
4+
#define Py_MEMORYOBJECT_H
5+
#ifdef __cplusplus
6+
extern "C" {
7+
#endif
8+
9+
PyAPI_DATA(PyTypeObject) PyMemoryView_Type;
10+
11+
#define PyMemoryView_Check(op) (Py_TYPE(op) == &PyMemoryView_Type)
12+
13+
/* Get a pointer to the underlying Py_buffer of a memoryview object. */
14+
#define PyMemoryView_GET_BUFFER(op) (&((PyMemoryViewObject *)(op))->view)
15+
/* Get a pointer to the PyObject from which originates a memoryview object. */
16+
#define PyMemoryView_GET_BASE(op) (((PyMemoryViewObject *)(op))->view.obj)
17+
18+
19+
PyAPI_FUNC(PyObject *) PyMemoryView_GetContiguous(PyObject *base,
20+
int buffertype,
21+
char fort);
22+
23+
/* Return a contiguous chunk of memory representing the buffer
24+
from an object in a memory view object. If a copy is made then the
25+
base object for the memory view will be a *new* bytes object.
26+
27+
Otherwise, the base-object will be the object itself and no
28+
data-copying will be done.
29+
30+
The buffertype argument can be PyBUF_READ, PyBUF_WRITE,
31+
PyBUF_SHADOW to determine whether the returned buffer
32+
should be READONLY, WRITABLE, or set to update the
33+
original buffer if a copy must be made. If buffertype is
34+
PyBUF_WRITE and the buffer is not contiguous an error will
35+
be raised. In this circumstance, the user can use
36+
PyBUF_SHADOW to ensure that a a writable temporary
37+
contiguous buffer is returned. The contents of this
38+
contiguous buffer will be copied back into the original
39+
object after the memoryview object is deleted as long as
40+
the original object is writable and allows setting an
41+
exclusive write lock. If this is not allowed by the
42+
original object, then a BufferError is raised.
43+
44+
If the object is multi-dimensional and if fortran is 'F',
45+
the first dimension of the underlying array will vary the
46+
fastest in the buffer. If fortran is 'C', then the last
47+
dimension will vary the fastest (C-style contiguous). If
48+
fortran is 'A', then it does not matter and you will get
49+
whatever the object decides is more efficient.
50+
51+
A new reference is returned that must be DECREF'd when finished.
52+
*/
53+
54+
PyAPI_FUNC(PyObject *) PyMemoryView_FromObject(PyObject *base);
55+
56+
PyAPI_FUNC(PyObject *) PyMemoryView_FromBuffer(Py_buffer *info);
57+
/* create new if bufptr is NULL
58+
will be a new bytesobject in base */
59+
60+
61+
/* The struct is declared here so that macros can work, but it shouldn't
62+
be considered public. Don't access those fields directly, use the macros
63+
and functions instead! */
64+
typedef struct {
65+
PyObject_HEAD
66+
PyObject *base;
67+
Py_buffer view;
68+
} PyMemoryViewObject;
69+
70+
71+
#ifdef __cplusplus
72+
}
73+
#endif
74+
#endif /* !Py_MEMORYOBJECT_H */

‎Include/object.h‎

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -159,21 +159,23 @@ typedef Py_ssize_t (*writebufferproc)(PyObject *, Py_ssize_t, void **);
159159
typedef Py_ssize_t (*segcountproc)(PyObject *, Py_ssize_t *);
160160
typedef Py_ssize_t (*charbufferproc)(PyObject *, Py_ssize_t, char **);
161161

162-
/* Py3k buffer interface */
163162

163+
/* Py3k buffer interface */
164164
typedef struct bufferinfo {
165-
void *buf;
166-
PyObject *obj; /* borrowed reference */
167-
Py_ssize_t len;
168-
Py_ssize_t itemsize; /* This is Py_ssize_t so it can be
169-
pointed to by strides in simple case.*/
170-
int readonly;
171-
int ndim;
172-
char *format;
173-
Py_ssize_t *shape;
174-
Py_ssize_t *strides;
175-
Py_ssize_t *suboffsets;
176-
void *internal;
165+
void *buf;
166+
PyObject *obj; /* owned reference */
167+
Py_ssize_t len;
168+
Py_ssize_t itemsize; /* This is Py_ssize_t so it can be
169+
pointed to by strides in simple case.*/
170+
int readonly;
171+
int ndim;
172+
char *format;
173+
Py_ssize_t *shape;
174+
Py_ssize_t *strides;
175+
Py_ssize_t *suboffsets;
176+
Py_ssize_t smalltable[2]; /* static store for shape and strides of
177+
mono-dimensional buffers. */
178+
void *internal;
177179
} Py_buffer;
178180

179181
typedef int (*getbufferproc)(PyObject *, Py_buffer *, int);

0 commit comments

Comments
 (0)