Skip to content

Some performance improvements for PyPy - #103

Merged
methane merged 7 commits into
msgpack:masterfrom
bwesterb:master
Jun 16, 2014
Merged

Some performance improvements for PyPy#103
methane merged 7 commits into
msgpack:masterfrom
bwesterb:master

Conversation

@bwesterb

Copy link
Copy Markdown
Contributor

No description provided.

@bwesterb

Copy link
Copy Markdown
Contributor Author

Ah, I see some tests failed. I will look intro that. For the moment, ignore the pull-request.

@bwesterb

Copy link
Copy Markdown
Contributor Author

Now it is ready to merge. The speed improvement is about 25% for my testcase. At the moment, pypy is three times slower than the C implementation.

@methane

methane commented Jun 16, 2014

Copy link
Copy Markdown
Member

Thanks.
I'm interested in performance difference between bytearray and manual buffering.
But for now, it's looks good to me

methane added a commit that referenced this pull request Jun 16, 2014
Some performance improvements for PyPy
@methane
methane merged commit 5cfa49b into msgpack:master Jun 16, 2014
@bwesterb

Copy link
Copy Markdown
Contributor Author

I've already looked into using memoryviews on strs instead of strs to prevent copying in _fb_read. That did not improve performance. I will look into byte arrays this week.

@methane

methane commented Jun 16, 2014

Copy link
Copy Markdown
Member

asyncio chose bytearray for receive buffer.
But Python 2's bytearray is not optimized for such usecase.

Python 2.7:

In [2]: x = bytearray(10000)

In [3]: %timeit del x[0]; x.append(1)
1000000 loops, best of 3: 327 ns per loop

In [4]: x = bytearray(100000)

In [5]: %timeit del x[0]; x.append(1)
100000 loops, best of 3: 4.78 µs per loop

Python 3.4:

In [1]: x = bytearray(10000)

In [2]: %timeit del x[0]; x.append(1)
10000000 loops, best of 3: 169 ns per loop

In [3]: x = bytearray(100000)

In [4]: %timeit del x[0]; x.append(1)
10000000 loops, best of 3: 167 ns per loop

PyPy 2.3.1:

In [1]: x = bytearray(10000)

In [2]: %timeit del x[0]; x.append(1)
100000 loops, best of 3: 7.28 µs per loop

In [3]: x = bytearray(100000)

In [4]: %timeit del x[0]; x.append(1)
10000 loops, best of 3: 71.5 µs per loop

@bwesterb

Copy link
Copy Markdown
Contributor Author

I reimplemented the buffer using bytearray. It is much slower than the current implementation.

https://github.com/bwesterb/msgpack-python/tree/bytearray

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants