divmod-users team mailing list archive

[Tristan Seligmann <mithrandi@xxxxxxxxxxxxx>]

I already wrote a blog post[1] with some benchmarking information about
Axiom on PyPy; in short, the speed-ups are reasonably significant, which is
nice. I thought I'd write a quick breakdown of the current status of the
porting efforts as well, hence this email.

Axiom trunk is already essentially fully functional on PyPy at this stage.
The only remaining test failures are highly unlikely to cause any real
application code to fail, and are dealt with in this branch (which deletes
the test in question, see the merge proposal for the explanation why):

https://code.launchpad.net/~divmod-dev/divmod.org/829869-explicit-type-dropping/+merge/72276

There is a second issue which currently does not result in any test
failures, but may result in running out of file descriptors in applications
that open and close stores very frequently:
https://bugs.launchpad.net/divmod-axiom/+bug/1208092

On my desktop machine, the test suite runs in 9.608s on CPython 2.7, and in
18.506s on PyPy 2.1, which is not too bad.

There is also a test failure on both CPython and PyPy that appears on newer
versions of SQLite:
https://code.launchpad.net/~divmod-dev/divmod.org/1209082-remove-bogus-performance-tests/+merge/178882

The Epsilon and Nevow test suites already pass on PyPy, which takes care of
all of the dependencies for Mantissa, with the exception of PyCrypto which
is used by twisted.conch. Unfortunately PyCrypto does not build on PyPy,
and as PyCrypto is unmaintained(?) it seems likely this won't be fixed any
time soon. However, this is really only needed for the SSH server in
Mantissa; a useful piece of functionality, to be sure, but something that
should be optional. Unfortunately the code does not conditionally import
conch/pycrypto: https://bugs.launchpad.net/mantissa/+bug/830343

Some initial work on making the #830343 allows the test suite to run, still
with a bunch of pycrypto-related failures. Ignoring these leaves us with:
some test failures also present on CPython (#1208384, #1208308), and a
failure in a historic upgrader test[2] that I haven't looked into yet, and
may just be caused by one of the other outstanding issues that needs fixing.

In summary, I'm very close to being able to get a green Mantissa test run
on PyPy, and from there trying out my Mantissa application on PyPy. I don't
yet have any performance figures for Mantissa on PyPy vs CPython, but once
things are more functional I may take a crack at this as well
(unfortunately there aren't any handy microbenchmarks like there are with
Axiom).

If you would like to help out, the easiest thing you can do is review one
of the branches currently up for review[3]. You might also be able to help
with #830357 / #1208308; the branch I currently have up[4] has a
functioning EndpointPort (I think), but still lacks test coverag and the
changes to make `axiomatic port` use EndpointPort instead of
strports.parse. Finally, untangling the pycrypto dependency is probably the
biggest remaining issue, but mostly a tedious one to solve rather than
being a "hard" problem.

[1] https://mithrandi.net/blog/2013/08/axiom-benchmark-results-on-pypy/

[2]
xmantissa.test.historic.test_privateApplication3to4.PrivateApplicationUpgradeTests.test_powerup

[3] https://code.launchpad.net/divmod.org/+activereviews

[4] https://code.launchpad.net/~divmod-dev/divmod.org/830357-endpoint-port/
-- 
mithrandi, i Ainil en-Balandor, a faer Ambar