Emulating Bash 'source' in Python

The problem with your approach is that you are trying to interpret bash scripts. First you just try to interpret the export statement. Then you notice people are using variable expansion. Later people will put conditionals in their files, or process substitutions. In the end you will have a full blown bash script interpreter with a gazillion bugs. Don't do that.

Let Bash interpret the file for you and then collect the results.

You can do it like this:

#! /usr/bin/env pythonimport osimport pprintimport shleximport subprocesscommand = shlex.split("env -i bash -c 'source init_env && env'")proc = subprocess.Popen(command, stdout = subprocess.PIPE)for line in proc.stdout:  (key, _, value) = line.partition("=")  os.environ[key] = valueproc.communicate()pprint.pprint(dict(os.environ))

Make sure that you handle errors in case bash fails to source init_env, or bash itself fails to execute, or subprocess fails to execute bash, or any other errors.

the env -i at the beginning of the command line creates a clean environment. that means you will only get the environment variables from init_env. if you want the inherited system environment then omit env -i.

Read the documentation on subprocess for more details.

Note: this will only capture variables set with the export statement, as env only prints exported variables.

Enjoy.

Note that the Python documentation says that if you want to manipulate the environment you should manipulate os.environ directly instead of using os.putenv(). I consider that a bug, but I digress.

Using pickle:

import os, pickle# For clarity, I moved this string out of the commandsource = 'source init_env'dump = '/usr/bin/python -c "import os,pickle;print pickle.dumps(os.environ)"'penv = os.popen('%s && %s' %(source,dump))env = pickle.loads(penv.read())os.environ = env

Updated:

This uses json, subprocess, and explicitly uses /bin/bash (for ubuntu support):

import os, subprocess as sp, jsonsource = 'source init_env'dump = '/usr/bin/python -c "import os, json;print json.dumps(dict(os.environ))"'pipe = sp.Popen(['/bin/bash', '-c', '%s && %s' %(source,dump)], stdout=sp.PIPE)env = json.loads(pipe.stdout.read())os.environ = env

Rather than having your Python script source the bash script, it would be simpler and more elegant to have a wrapper script source init_env and then run your Python script with the modified environment.

#!/bin/bashsource init_env/run/python/script.py