IPython uses a different $PATH environment than the shell
To summarize and complement @holdenweb's helpful answer, particularly with respect to OS X:
- A subshell started from IPython with
!
is a non-interactive non-login instance of the user's default shell - even if IPython was started from a different shell.- Behind the scenes, the subshell is started with
path/to/default/shell -c ...
- To see specifics, run
!ps -p $$ && :
- Behind the scenes, the subshell is started with
echo $SHELL
always tells you the default shell - even when run from a different shell.Initialization files sourced in non-interactive non-login shells:
zsh:
:/etc/zshenv
and~/.zshenv
bash
: a script pointed to in the$BASH_ENV
variable, if defined.
As pointed out, shells load different/additional initialization files depending on whether:
- the shell is a login shell or not
- an interactive shell or not
Note that a login shell can be interactive or not, and an interactive shell can be a login shell or not.
Thus, in the case at hand, potentially two additional initialization files were loaded in the interactive shell, explaining the difference in behavior between the interactive shell and the subshell created by IPython:
~/.zprofile
- if the shell is a login shell - which will be the case ifzsh
is the default shell (on OS X, all interactive instances of the default shell created in a terminal such asTerminal.app
are login shells).~/.zshrc
Finally, on a related note, note that on OS X the default $PATH
for NON-shell processes is:
/usr/bin:/bin:/usr/sbin:/sbin # Note the absence of /usr/local/bin.
and only shells add /usr/local/bin
(by default; extensible) to that, via system-wide initialization files (that call /usr/libexec/path_helper
):
zsh
:/etc/zshenv
- Note: takes effect for ALL
zsh
instances.
bash
(also when invoked assh
),ksh
:/etc/profile
- Note: takes effect for LOGIN shells only.
Upshot:
- Non-shell apps see only the default
$PATH
(notably, without/usr/local/bin
and other additions made via shell initialization files). bash
/sh
andksh
non-interactive non-login shells not launched from a login shell also see only the default$PATH
. This happens, for instance, when GUI command-line launchers such as Alfred create shell instances.zsh
is not affected, because/etc/zshenv
is read by everyzsh
instance.
First of all, sys.path
has little to do with this; it's just the list of locations in which the Python interpreter can look when importing modules, and does not determine where executable programs are found by the shell. Some elements of the PATH
are seen in that list, though, because the interpreter uses the path of its executable to build certain entries on sys.path
.
os.environ['PATH']
is, unsurprisingly, the same as the $PATH
variable from the environment in which IPython is running. !echo $PATH
prints out the $PATH
variable from the subshell started by IPython to execute the shell escape (!
).
One possible cause is that IPython is executing the shell commands using the OS standard shell which has not been tailored to set its path in the same way that your zsh
has. You can confirm this by executing the IPython command !echo $SHELL
. Since you confirmed this was not the case then the difference can be accounted for by the differences between a login shell and an interactive shell.
I don't know of a configuration item that will tell IPython to use another shell, but it's possible there is one. As a workaround, just make sure the other shell has a correctly configured path, or that interactive shells also see the environment you require.
See @mklement0's extremely authoritative answer for the whole, gory, messy details.