Sibling package imports
Tired of sys.path hacks?
There are plenty of
sys.path.append -hacks available, but I found an alternative way of solving the problem in hand.
- Wrap the code into one folder (e.g.
setup.pyscript where you use setuptools.setup(). (see minimal
- Pip install the package in editable state with
pip install -e <myproject_folder>
- Import using
from packaged_stuff.modulename import function_name
The starting point is the file structure you have provided, wrapped in a folder called
.└── myproject ├── api │ ├── api_key.py │ ├── api.py │ └── __init__.py ├── examples │ ├── example_one.py │ ├── example_two.py │ └── __init__.py ├── LICENCE.md ├── README.md └── tests ├── __init__.py └── test_one.py
I will call the
. the root folder, and in my example case it is located at
As a test case, let's use the following ./api/api.py
def function_from_api(): return 'I am the return value from api.api!'
from api.api import function_from_apidef test_function(): print(function_from_api())if __name__ == '__main__': test_function()
Try to run test_one:
PS C:\tmp\test_imports> python .\myproject\tests\test_one.pyTraceback (most recent call last): File ".\myproject\tests\test_one.py", line 1, in <module> from api.api import function_from_apiModuleNotFoundError: No module named 'api'
Also trying relative imports wont work:
from ..api.api import function_from_api would result into
PS C:\tmp\test_imports> python .\myproject\tests\test_one.pyTraceback (most recent call last): File ".\tests\test_one.py", line 1, in <module> from ..api.api import function_from_apiValueError: attempted relative import beyond top-level package
- Make a setup.py file to the root level directory
The contents for the
setup.py would be*
from setuptools import setup, find_packagessetup(name='myproject', version='1.0', packages=find_packages())
- Use a virtual environment
If you are familiar with virtual environments, activate one, and skip to the next step. Usage of virtual environments are not absolutely required, but they will really help you out in the long run (when you have more than 1 project ongoing..). The most basic steps are (run in the root folder)
- Create virtual env
python -m venv venv
- Activate virtual env
source ./venv/bin/activate(Linux, macOS) or
To learn more about this, just Google out "python virtual env tutorial" or similar. You probably never need any other commands than creating, activating and deactivating.
Once you have made and activated a virtual environment, your console should give the name of the virtual environment in parenthesis
PS C:\tmp\test_imports> python -m venv venvPS C:\tmp\test_imports> .\venv\Scripts\activate(venv) PS C:\tmp\test_imports>
and your folder tree should look like this**
.├── myproject│ ├── api│ │ ├── api_key.py│ │ ├── api.py│ │ └── __init__.py│ ├── examples│ │ ├── example_one.py│ │ ├── example_two.py│ │ └── __init__.py│ ├── LICENCE.md│ ├── README.md│ └── tests│ ├── __init__.py│ └── test_one.py├── setup.py└── venv ├── Include ├── Lib ├── pyvenv.cfg └── Scripts [87 entries exceeds filelimit, not opening dir]
- pip install your project in editable state
Install your top level package
pip. The trick is to use the
-e flag when doing the install. This way it is installed in an editable state, and all the edits made to the .py files will be automatically included in the installed package.
In the root directory, run
pip install -e . (note the dot, it stands for "current directory")
You can also see that it is installed by using
(venv) PS C:\tmp\test_imports> pip install -e .Obtaining file:///C:/tmp/test_importsInstalling collected packages: myproject Running setup.py develop for myprojectSuccessfully installed myproject(venv) PS C:\tmp\test_imports> pip freezemyproject==1.0
myproject.into your imports
Note that you will have to add
myproject. only into imports that would not work otherwise. Imports that worked without the
pip install will work still work fine. See an example below.
Test the solution
Now, let's test the solution using
api.py defined above, and
test_one.py defined below.
from myproject.api.api import function_from_apidef test_function(): print(function_from_api())if __name__ == '__main__': test_function()
running the test
(venv) PS C:\tmp\test_imports> python .\myproject\tests\test_one.pyI am the return value from api.api!
* See the setuptools docs for more verbose setup.py examples.
** In reality, you could put your virtual environment anywhere on your hard disk.
Seven years after
Since I wrote the answer below, modifying
sys.path is still a quick-and-dirty trick that works well for private scripts, but there has been several improvements
- Installing the package (in a virtualenv or not) will give you what you want, though I would suggest using pip to do it rather than using setuptools directly (and using
setup.cfgto store the metadata)
- Using the
-mflag and running as a package works too (but will turn out a bit awkward if you want to convert your working directory into an installable package).
- For the tests, specifically, pytest is able to find the api package in this situation and takes care of the
sys.pathhacks for you
So it really depends on what you want to do. In your case, though, since it seems that your goal is to make a proper package at some point, installing through
pip -e is probably your best bet, even if it is not perfect yet.
As already stated elsewhere, the awful truth is that you have to do ugly hacks to allow imports from siblings modules or parents package from a
__main__ module. The issue is detailed in PEP 366. PEP 3122 attempted to handle imports in a more rational way but Guido has rejected it one the account of
The only use case seems to be running scripts that happen to be living inside a module's directory, which I've always seen as an antipattern.
Though, I use this pattern on a regular basis with
# Ugly hack to allow absolute import from the root folder# whatever its name is. Please forgive the heresy.if __name__ == "__main__" and __package__ is None: from sys import path from os.path import dirname as dir path.append(dir(path)) __package__ = "examples"import api
path is your running script's parent folder and
dir(path) your top level folder.
I have still not been able to use relative imports with this, though, but it does allow absolute imports from the top level (in your example
api's parent folder).