How do you get PyPy, Django and PostgreSQL to work together?

psycopg2cffi (Updated 2015)

psycopg2cffi is yet another psycopg2-compatible replacement and should provide the best PostgreSQL performance with PyPy. Add this to your settings.py to remain compatible with both:

try:    import psycopg2except ImportError:    # Fall back to psycopg2cffi    from psycopg2cffi import compat    compat.register()

psycopg2-ctypes (2012)

I also know that some people are using psycopg2-ctypes.

This is the easiest way; to stay compatible with both, just add this code in your Django settings.py:

try:    import psycopg2except ImportError:    # Fall back to psycopg2-ctypes    from psycopg2ct import compat    compat.register()

I tested this a few releases ago; sadly in my experience, psycopg2-ctypes negates the small performance gains afforded by PyPy. But YMMV, it depends on how JIT-friendly your code is in general and what fraction of time you actually spend running Python code. And maybe PyPy has just improved since then.

and I don't think psycopg2-ctypes is ready for Windows yet

I haven't tried this, but ctypes is platform-independent. AFAICT you just have to make sure that the libpq.dll library is loadable (located in a directory in your PATH environment variable or local directory) and it should work on Windows just like in Linux.

pypy-postgresql

I do see that Alex Gaynor has made a fork of PyPy called pypy-postgresql.

I don't think this is a good choice in the long term. The branch hasn't been updated for more than a year and my attempts to build it have failed. And it seems wrong to hard-code a PostgreSQL driver in the interpreter anyway.

I believe there are no binaries out there of pypy-postgresql either, so if you want to use it, you'd need to build the whole PyPy branch yourself. Not for the faint of heart: it takes tens of minutes and a machine with at least 4 GB of memory. (Official instructions: http://pypy.org/download.html#building-from-source)

To build, you first need the source. If you have Mercurial installed, you can simply hg clone https://bitbucket.org/alex_gaynor/pypy-postgresql. If not, you can download the automagic "tip" zip file: https://bitbucket.org/alex_gaynor/pypy-postgresql/get/tip.zip

Open a command line, go into the decompressed directory, and then inside pypy/translator/goal

If you have PyPy installed, it's recommended to use that for building:

pypy translate.py -Ojit

Otherwise:

python translate.py -Ojit

Sadly this is where my knowledge ends. I get the error "BytecodeCorruption: unimplemented opcode, ofs=234, code=203, name=BUILD_LIST_FROM_ARG"

Some additional resources:

• PyPy compatibility information: DB adaptors
• PostgreSQL page on the Python wiki
• psycopg2cffi by Konstantin Lopuhin:
  cffi based implementation of psycopg2 for PyPy 2.0 and newer
  (blog post, GitHub repo, PyPI page, pypy-dev thread)
  – this looks like the strongest candidate currently, but I haven't tested it yet
• psycopg2ct by Michael van Tellingen:
  ctypes based implementation of psycopg2 for PyPy 1.6 and newer
  (GitHub repo, PyPI page)
• pypy-postgresql by Alex Gaynor:
  abandoned RPython port of psycopg2 implemented as a fork of PyPy (Bitbucket repo)
• pypq:
  "Python PostgreSQL DBAPI 2.0 compliant driver using ctypes and libpq.so, works with PyPy"
  (discussion, PyPI page)
• bpgsql:
  "Barebones pure-python PostGreSQL client. Mostly DB-API 2.0 (PEP 249) compliant. Includes an experimental Django 1.0 backend"
  (discussion, web page, Google Code page)
• pg8000:
  "a DB-API 2.0 compatible Pure-Python interface to the PostgreSQL database engine [...] does not rely on any external libraries (such as a compiled python module, or PostgreSQL’s libpq library)"
  (web page, GitHub repo, PyPI page)