Twisted + SQLAlchemy and the best way to do it Twisted + SQLAlchemy and the best way to do it database database

Twisted + SQLAlchemy and the best way to do it


In the intervening couple of years, Alex Gaynor created https://github.com/alex/alchimia which may be a better central repository for doing integration with SQLAlchemy and Twisted.


There's a storm branch that you can use with twisted directly (internally it does the defer to thread stuff) on launchpad https://code.launchpad.net/~therve/storm/twisted-integration. I've used it nicely.

Sadly sqlalchemy is significantly more complex in implementation to audit for async usage. If you really want to use it, i'd recommend an out of process approach with a storage rpc layer.

alternatively if your feeling adventurous and using postgresql, the latest pyscopg2 supports true async usage (https://launchpad.net/txpostgres), and the storm source is pretty simple to hack on ;-)

incidentally the storm you tried last year may not have had the C-extension on by default (it is now in the latest releases.) which might account for your speed issues.


Firstly, I can unfortunately only second your opinion that twisted andSQLAlchemy don't play along very well. I have worked some with bothand would be somewhat afraid of the complexity that would arise fromputting them together.

All the database integration layers that I know of to date usetwisteds threading integration layer, and if you want to avoid that atall costs you are pretty much stuck with point 4 in your list.

On the other hand, I have seen examples of database connecting codeusing deferToThread() and friends that worked very well.

Anyway, some pointers if you'd be ready to consider other frameworksthan SQLAlchemy:

The DivMod guys have been doing some tentative work on twisted -database integration based on the Storm ORM (google for "storm orm").

See this link for an example:

http://divmod.readthedocs.org/en/latest/products/nevow/storm-approach.html

Also, head over to DivMod's site and have a look at the sources oftheir Axiom db layer (probably not of any use to you directly sinceit's Sqlite only, but it's principles might be useful).