Reference requirements.txt for the install_requires kwarg in setuptools setup.py file
You can flip it around and list the dependencies in
setup.py and have a single character — a dot
. — in
Alternatively, even if not advised, it is still possible to parse the
requirements.txt file (if it doesn't refer any external requirements by URL) with the following hack (tested with
install_reqs = parse_requirements('requirements.txt', session='hack')
This doesn't filter environment markers though.
In old versions of pip, more specifically older than 6.0, there is a public API that can be used to achieve this. A requirement file can contain comments (
#) and can include some other files (
-r). Thus, if you really want to parse a
requirements.txt you can use the pip parser:
from pip.req import parse_requirements# parse_requirements() returns generator of pip.req.InstallRequirement objectsinstall_reqs = parse_requirements(<requirements_path>)# reqs is a list of requirement# e.g. ['django==1.5.1', 'mezzanine==1.4.6']reqs = [str(ir.req) for ir in install_reqs]setup( ... install_requires=reqs)
On the face of it, it does seem that
setup.py are silly duplicates, but it's important to understand that while the form is similar, the intended function is very different.
The goal of a package author, when specifying dependencies, is to say "wherever you install this package, these are the other packages you need, in order for this package to work."
In contrast, the deployment author (which may be the same person at a different time) has a different job, in that they say "here's the list of packages that we've gathered together and tested and that I now need to install".
The package author writes for a wide variety of scenarios, because they're putting their work out there to be used in ways they may not know about, and have no way of knowing what packages will be installed alongside their package. In order to be a good neighbor and avoid dependency version conflicts with other packages, they need to specify as wide a range of dependency versions as can possibly work. This is what
The deployment author writes for a very different, very specific goal: a single instance of an installed application or service, installed on a particular computer. In order to precisely control a deployment, and be sure that the right packages are tested and deployed, the deployment author must specify the exact version and source-location of every package to be installed, including dependencies and dependency's dependencies. With this spec, a deployment can be repeatably applied to several machines, or tested on a test machine, and the deployment author can be confident that the same packages are deployed every time. This is what a
So you can see that, while they both look like a big list of packages and versions, these two things have very different jobs. And it's definitely easy to mix this up and get it wrong! But the right way to think about this is that
requirements.txt is an "answer" to the "question" posed by the requirements in all the various
setup.py package files. Rather than write it by hand, it's often generated by telling pip to look at all the
setup.py files in a set of desired packages, find a set of packages that it thinks fits all the requirements, and then, after they're installed, "freeze" that list of packages into a text file (this is where the
pip freeze name comes from).
So the takeaway:
setup.pyshould declare the loosest possible dependency versions that are still workable. Its job is to say what a particular package can work with.
requirements.txtis a deployment manifest that defines an entire installation job, and shouldn't be thought of as tied to any one package. Its job is to declare an exhaustive list of all the necessary packages to make a deployment work.
- Because these two things have such different content and reasons for existing, it's not feasible to simply copy one into the other.
- install_requires vs Requirements files from the Python packaging user guide.
It can't take a file handle. The
install_requires argument can only be a string or a list of strings.
You can, of course, read your file in the setup script and pass it as a list of strings to
import osfrom setuptools import setupwith open('requirements.txt') as f: required = f.read().splitlines()setup(...install_requires=required,...)