What's the closest I can get to calling a Python function using a different Python version? What's the closest I can get to calling a Python function using a different Python version? python python

What's the closest I can get to calling a Python function using a different Python version?


Here is a complete example implementation using subprocess and pickle that I actually tested. Note that you need to use protocol version 2 explicitly for pickling on the Python 3 side (at least for the combo Python 3.5.2 and Python 2.7.3).

# py3bridge.pyimport sysimport pickleimport importlibimport ioimport tracebackimport subprocessclass Py3Wrapper(object):    def __init__(self, mod_name, func_name):        self.mod_name = mod_name        self.func_name = func_name    def __call__(self, *args, **kwargs):        p = subprocess.Popen(['python3', '-m', 'py3bridge',                              self.mod_name, self.func_name],                              stdin=subprocess.PIPE,                              stdout=subprocess.PIPE)        stdout, _ = p.communicate(pickle.dumps((args, kwargs)))        data = pickle.loads(stdout)        if data['success']:            return data['result']        else:            raise Exception(data['stacktrace'])def main():    try:        target_module = sys.argv[1]        target_function = sys.argv[2]        args, kwargs = pickle.load(sys.stdin.buffer)        mod = importlib.import_module(target_module)        func = getattr(mod, target_function)        result = func(*args, **kwargs)        data = dict(success=True, result=result)    except Exception:        st = io.StringIO()        traceback.print_exc(file=st)        data = dict(success=False, stacktrace=st.getvalue())    pickle.dump(data, sys.stdout.buffer, 2)if __name__ == '__main__':    main()

The Python 3 module (using the pathlib module for the showcase)

# spam.pyimport pathlibdef listdir(p):    return [str(c) for c in pathlib.Path(p).iterdir()]

The Python 2 module using spam.listdir

# beans.pyimport py3bridgedelegate = py3bridge.Py3Wrapper('spam', 'listdir')py3result = delegate('.')print py3result


Assuming the caller is Python3.5+, you have access to a nicer subprocess module. Perhaps you could user subprocess.run, and communicate via pickled Python objects sent through stdin and stdout, respectively. There would be some setup to do, but no parsing on your side, or mucking with strings etc.

Here's an example of Python2 code via subprocess.Popen

p = subprocess.Popen(python3_args, stdin=subprocess.PIPE, stdout=subprocess.PIPE)stdout, stderr = p.communicate(pickle.dumps(python3_args))result = pickle.load(stdout)


You could create a simple script as such :

import sysimport my_wrapped_moduleimport jsonparams = sys.argvscript = params.pop(0)function = params.pop(0)print(json.dumps(getattr(my_wrapped_module, function)(*params)))

You'll be able to call it like that :

pythonx.x wrapper.py myfunction param1 param2

This is obviously a security hazard though, be careful.

Also note that if your params are anything else than string or integers, you'll have some issues, so maybe think about transmitting params as a json string, and convert it using json.loads() in the wrapper.