How to capture network traffic using selenium webdriver and browsermob proxy on Python? How to capture network traffic using selenium webdriver and browsermob proxy on Python? selenium selenium

How to capture network traffic using selenium webdriver and browsermob proxy on Python?


Just stumbled across this project https://github.com/derekargueta/selenium-profiler. Spits out all network data for a URL. Shouldn't be hard to hack and integrate into whatever tests you're running.

Original source: https://www.openhub.net/p/selenium-profiler


For me, following code component works just fine.

profile = webdriver.FirefoxProfile()profile.set_proxy(proxy.selenium_proxy())driver = webdriver.Firefox(firefox_profile=profile)


I am sharing my solution, this would not write any logs to any filebut you can collect all sort of messages such as Errors, Warnings, Logs, Info, Debug , CSS, XHR as well as Requests(traffic)1. We are going to create Firefox profile so that we can enable option of "Persist Logs" on Firefox (you can try it to enable on your default browser and see if it launches with "Persist Logs" without creating firefox profile )2. we need to modify the Firefox initialize code where this line will do magic : options.AddArgument("--jsconsole");so complete Selenium Firefox code would be,  this will open Browser Consoleeverytime you execute your automation : else if (browser.Equals(Constant.Firefox))            {                var profileManager = new FirefoxProfileManager();                FirefoxProfile profile = profileManager.GetProfile("ConsoleLogs");                FirefoxDriverService service = FirefoxDriverService.CreateDefaultService(DrivePath);                service.FirefoxBinaryPath = DrivePath;                                profile.SetPreference("security.sandbox.content.level", 5);                profile.SetPreference("dom.webnotifications.enabled", false);                profile.AcceptUntrustedCertificates = true;                FirefoxOptions options = new FirefoxOptions();                options.AddArgument("--jsconsole");                options.AcceptInsecureCertificates = true;                options.Profile = profile;                options.SetPreference("browser.popups.showPopupBlocker", false);                driver = new FirefoxDriver(service.FirefoxBinaryPath, options, TimeSpan.FromSeconds(100));                driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(10);                           }3. Now you can write your logic since you have traffic/ logging window open so don't go to next execution if test fails. That way Browser Console will keep your errors messages and help you to troubleshoot further Browser : Firefox  v 61 How can you launch Browser Console for firefox:1. open firefox (and give any URL )2. Press Ctrl+Shift+J (or Cmd+Shift+J on a Mac) Link : https://developer.mozilla.org/en-US/docs/Tools/Browser_Console