How do I set a proxy for phantomjs/ghostdriver in python webdriver? How do I set a proxy for phantomjs/ghostdriver in python webdriver? python python

How do I set a proxy for phantomjs/ghostdriver in python webdriver?


Below is the example of how to set proxy for PhantomJs in Python. You may change proxy type: socks5/http.

service_args = [    '--proxy=127.0.0.1:9999',    '--proxy-type=socks5',    ]browser = webdriver.PhantomJS('../path_to/phantomjs',service_args=service_args)


I dug a little and I found that the functionality is there, but it is not exposed. So it requires a handy monkey wrench to patch it up. Here is the solution that works for me until this functionality is fully exposed in the webdriver call.

EDIT: it seems the service_args are now exposed, you no longer need to monkey patch selenium to use the proxy ... see @alex-czech answer for how to use.

from selenium import webdriverfrom selenium.webdriver.phantomjs.service import Service as PhantomJSServicephantomjs_path = '/usr/lib/node_modules/phantomjs/lib/phantom/bin/phantomjs'# monkey patch Service temporarily to include desired argsclass NewService(PhantomJSService):    def __init__(self, *args, **kwargs):        service_args = kwargs.setdefault('service_args', [])        service_args += [            '--proxy=localhost:8080',            '--proxy-type=http',        ]        super(NewService, self).__init__(*args, **kwargs)webdriver.phantomjs.webdriver.Service = NewService# init the webdriverself.driver = webdriver.PhantomJS(phantomjs_path)# undo monkey patchwebdriver.phantomjs.webdriver.Service = PhantomJSService

Also useful are the following settings, especially when using a proxy that may take a very long time to load.

max_wait = 60self.driver.set_window_size(1024, 768)self.driver.set_page_load_timeout(max_wait)self.driver.set_script_timeout(max_wait)


The following is how to do the same with the Webdriver in Ruby. I couldn't find this anywhere online until I dug into the source code:

phantomjs_args = [ '--proxy=127.0.0.1:9999', '--proxy-type=socks5']phantomjs_caps = { "phantomjs.cli.args" => phantomjs_args }driver = Selenium::WebDriver.for(:phantomjs, :desired_capabilities => phantomjs_caps)