How to avoid building C library with my python package? How to avoid building C library with my python package? python python

How to avoid building C library with my python package?


You're certainly heading down the right path according to my research... As Daniel says, the only option you have is to build and distribute the binaries yourself.

In general, the recommended way to install packages is covered well in the packaging user guide. I won't repeat advice there as you have clearly already found it. However the key point in there is that the Python community, specifically PyPA are trying to standardize on using platform wheels to package binary extensions. Sadly, there are a few issues at this point:

  1. You cannot create distributions for all Linux variants, but you can build wheels for a compatible subset. See https://www.python.org/dev/peps/pep-0513/ for details.
  2. The advice on building extensions is somewhat incomplete, reflecting the lack of a complete solution for binary distributions.
  3. People then try to build their own library and distribute it as a data file, which confuses setuptools.

I think you are hitting this last issue. A workaround is to force the Distribution to build a platform wheel by overriding is_pure() to always return False. However you could just keep your original build instructions and bdist_wheel should handle it.

Once you've built the wheel, though, you still need to distribute it and maybe other binary packages that it uses or use it. At this point, you probably need to use one of the recommended tools like conda or a PyPI proxy like devpi to serve up your wheels.

EDIT: To answer the extra question about cross-compiling

As covered here Python 2.6 and later allows cross-compilation for Windows 32/64-bit builds. There is no formal support for other packages on other platforms and people have had limited success trying to do it. You are really best off building natively on each of your Linux/Mac/Windows environments.


@rth and @PeterBrittain help me a lot. Here the solution I use:

structure folder :

setup.pypython_package/    lib/        libfoo.dylib        libfoo.dll    __init__.py    main.py

setup.py :

from setuptools import setup, distclass BinaryDistribution(dist.Distribution):    def is_pure(self):        return Falsesetup(    name='python_package',    package_data={'python_package': ['lib/libfoo.dylib','lib/libfoo.dll']},    include_package_data=True,    distclass=BinaryDistribution,    packages=['python_package'],)

main.py :

#!/usr/bin/env pythonimport platformfrom ctypes import CDLL, c_char_pimport pkg_resourcessysname = platform.system()if sysname == 'Darwin':    lib_name = "libfoo.dylib"elif sysname == 'Windows':    lib_name = "libfoo.dll"else:    lib_name = "libfoo.so"lib_path = pkg_resources.resource_filename('python_package', 'lib/{}'.format(lib_name))foo = CDLL(lib_path)bar = foo.barbar.restype = c_char_pbar.argtypes = [c_char_p]print(bar('hello'))

build wheel :

python setup.py bdist_wheel

It creates a specific plateform wheel rtfdoc-0.0.1-cp34-cp34m-macosx_10_10_x86_64.whl and mac users can do a simple pip install rtfdoc-0.0.1-cp34-cp34m-macosx_10_10_x86_64.whl

This solution is not entirely satisfactory:

  • I think I will still need to create a Extension or look at SCons for linux users.
  • Update my package will be difficult because of the production of the lib
  • I don't know how to manage .dll for 32 bit and 64 bit

Thank you, I learned a lot, and I understand why setup.py in Pillow or Psycopg2 are huge


You can use cibuildwheel to build wheels on Travis CI and/or Appveyor for all kinds of platforms and Python versions. This tool can also deploy your wheels on PyPI or elsewhere.