Extending python - to swig, not to swig or Cython

You should consider Boost.Python if you are not planning to generate bindings for other languages as well with swig.

If you have a lot of functions and classes to bind, Py++ is a great tool that automatically generates the needed code to make the bindings.

Pybindgen may also be an option, but it's a new project and less complete that Boost.Python.

Edit:

Maybe I need to be more explicit about pro and cons.

• Swig:

  pro: you can generate bindings for many scripting languages.

  cons: I don't like the way the parser works. I don't know if the made some progress but two years ago the C++ parser was quite limited. Most of the time I had to copy/past my .h headers add some % characters and give extra hints to the swig parser.

  I was also needed to deal with the Python C-API from time to time for (not so) complicated type conversions.

  I'm not using it anymore.

• Boost.Python:

  pro:It's a very complete library. It allows you to do almost everything that is possible with the C-API, but in C++. I never had to write C-API code with this library. I also never encountered bug due to the library. Code for bindings either works like a charm or refuse compile.

  It's probably one of the best solutions currently available if you already have some C++ library to bind. But if you only have a small C function to rewrite, I would probably try with Cython.

  cons: if you don't have a pre-compiled Boost.Python library you're going to use Bjam (sort of make replacement). I really hate Bjam and its syntax.

  Python libraries created with B.P tend to become obese. It also takes a lot of time to compile them.

• Py++ (discontinued): it's Boost.Python made easy. Py++ uses a C++ parser to read your code and then generates Boost.Python code automatically. You also have a great support from its author (no it's not me ;-) ).

  cons: only the problems due to Boost.Python itself. Update: As of 2014 this project now looks discontinued.

• Pybindgen:

  It generates the code dealing with the C-API. You can either describe functions and classes in a Python file, or let Pybindgen read your headers and generate bindings automatically (for this it uses pygccxml, a python library wrote by the author of Py++).

  cons: it's a young project, with a smaller team than Boost.Python. There are still some limitations: you cannot use multiple inheritance for your C++ classes, Callbacks (not automatically, custom callback handling code can be written, though). Translation of Python exceptions to C.

  It's definitely worth a good look.

• A new one:On 2009/01/20 the author of Py++ announced a new package for interfacing C/C++ code with python. It is based on ctypes. I didn't try it already but I will! Note: this project looks discontiued, as Py++.

• CFFI: I did not know the existence of this one until very recently so for now I cannot give my opinion. It looks like you can define C functions in Python strings and call them directly from the same Python module.

• Cython: This is the method I'm currently using in my projects. Basically you write code in special .pyx files. Those files are compiled (translated) into C code which in turn are compiled to CPython modules. Cython code can look like regular Python (and in fact pure Python are valid .pyx Cython files), but you can also more information like variable types. This optional typing allows Cython to generate faster C code. Code in Cython files can call both pure Python functions but also C and C++ functions (and also C++ methods).

  It took me some time to think in Cython, that in the same code call C and C++ function, mix Python and C variables, and so on. But it's a very powerful language, with an active (in 2014) and friendly community.

SWIG 2.0.4 has introduced a new -builtin option that improves performance.I did some benchmarking using an example program that does a lot of fast calls to a C++ extension.I built the extension using boost.python, PyBindGen, SIP and SWIG with and without the -builtin option. Here are the results (average of 100 runs):

SWIG with -builtin     2.67sSIP                    2.70sPyBindGen              2.74sboost.python           3.07sSWIG without -builtin  4.65s

SWIG used to be slowest. With the new -builtin option, SWIG seems to be fastest.

For sure you will always have a performance gain doing this by hand, but the gain will be very small compared to the effort required to do this. I don't have any figure to give you but I don't recommend this, because you will need to maintain the interface by hand, and this is not an option if your module is large!

You did the right thing to chose to use a scripting language because you wanted rapid development. This way you've avoided the early optimization syndrome, and now you want to optimize bottleneck parts, great! But if you do the C/python interface by hand you will fall in the early optimization syndrome for sure.

If you want something with less interface code, you can think about creating a dll from your C code, and use that library directly from python with cstruct.

Consider also Cython if you want to use only python code in your program.