PyPy compatibility

See original GitHub issue

This was raised on twitter, and may be important for some users. Currently msgspec is written as a c extension using some private cpython apis, making it incompatible with pypy. It would be good to have a pypy compatible build, whether through changes to the c extension or through a separate pure python release. Either seems fine to me, provided the result doesn’t decrease cpython performance noticeably.

Issue Analytics

State:
Created 3 years ago
Comments:5 (3 by maintainers)

Top GitHub Comments

2reactions

jcristcommented, Dec 13, 2022

This is not something I plan on supporting in a volunteer-only basis (if someone wanted to pay for this, I might be willing to expand support).

A few reasons:

msgspec is heavily tied to the CPython c-api for performance. PyPy does support a subset of the CPython c-api, but it does so in a way that is not very performant, and isn’t 100% compatible. We could add shims everywhere that’s needed to make msgspec PyPy compatible, but that would muddy up the code, increasing our maintenance burden.
Due to how PyPy’s c-api layer works, performance on PyPy would be worse than on CPython. Whether this matters is application dependent.
CPython is getting faster, reducing the advantages of PyPy
Other “fast” libraries (e.g. asyncpg, orjson, …) also don’t support PyPy, and people generally seem fine with that.

0reactions

Goldzihercommented, Dec 13, 2022

This is not something I plan on supporting in a volunteer-only basis (if someone wanted to pay for this, I might be willing to expand support).

A few reasons:

msgspec is heavily tied to the CPython c-api for performance. PyPy does support a subset of the CPython c-api, but it does so in a way that is not very performant, and isn’t 100% compatible. We could add shims everywhere that’s needed to make msgspec PyPy compatible, but that would muddy up the code, increasing our maintenance burden.

Due to how PyPy’s c-api layer works, performance on PyPy would be worse than on CPython. Whether this matters is application dependent.

CPython is getting faster, reducing the advantages of PyPy

Other “fast” libraries (e.g. asyncpg, orjson, …) also don’t support PyPy, and people generally seem fine with that.

Thanks for clarifying this. I think we are fine not having pypy support.

Top Results From Across the Web

Python compatibility - PyPy

Python compatibility. The goal of this page is to point out some of the differences between running python with PyPy and with CPython ......

PyPy - packages

PyPy's Python packages compatibility. PyPy is a fast, compliant alternative implementation of the Python language. Nearly all Python packages install ...

Frequently Asked Questions - PyPy documentation

Which Python version (2.x?) does PyPy implement?¶. PyPy comes in two versions: one is fully compatible with Python 2.7;; the other is fully ......

PyPy3 v5.5.0 - PyPy documentation

Coming four months after PyPy3.3 v5.2, it improves compatibility with Python 3.3 (3.3.5). We strongly recommend updating from previous PyPy3 versions.

PyPy v7.3.6: release of python 2.7, 3.7, and 3.8

The interpreters are based on much the same codebase, thus the multiple release. This is a micro release, all APIs are compatible with...