Sharing a database between Twisted and Django Sharing a database between Twisted and Django database database

Sharing a database between Twisted and Django


First of all I'd identify why you need both Django and Twisted. Assuming you are comfortable with Twisted using twisted.web and auth will easily be sufficient and you'll be able to reuse your database layer for both the frontend and backend apps.

Alternatively you could look at it the other way, what is Twisted doing better as a game server? Are you hoping to support more players (more simultaneous connections) or something else? Consider that if you must use threaded within twisted to do blocking database access that you are most likely not going to be able to efficently/reliably support hundreds of simultaneous threads. Remember python has a Global Interpreter Lock so threads are not necessarily the best way to scale.

You should also consider why you are looking to use a SQL Database and an ORM. Does your game have data that is really best suited to being stored in an relational database? Perhaps it's worth examining something like MongoDB or another key-value or object database for storing game state. Many of these NoSQL stores have both blocking drivers for use in Django and non-blocking drivers for use in Twisted (txmongo for example).

That said, if you're dead set on using both Django and Twisted there are a few techniques for embedding blocking DB access into a non-blocking Twisted server.

  1. adbapi (uses twisted thread pool)
  2. Direct use of the twisted thread pool using reactor.deferToThread
  3. The Storm ORM has a branch providing Twisted support (it handles deferToThread calls internally)
  4. SAsync is a library that tries to make SQLAlchemy work in an Async way
  5. Have twisted interact via RPC with a process that manages the blocking DB

So you should be able to manage the Django ORM objects yourself by importing them in twisted and being very careful making calls to reactor.deferToThread. There are many possible issues when working with these objects within twisted in that some ORM objects can issue SQL when accessing/setting a property, etc.

I realize this isn't necessarily the answer you were expecting but perhaps more detail about what you're hoping to accomplish and why you are choosing these specific technologies will allow folks to get you better answers.


I would just avoid the Django ORM, it's not all that and it would be a pain to access outside of a Django context (witness the work that was required to make Django support multiple databases). Twisted database access always requires threads (even with twisted.adbapi), and threads give you access to any ORM you choose. SQLalchemy would be a good choice.