Selenium and BeautifulSoup: sharing and pulling session data resources to multiple libraries in python Selenium and BeautifulSoup: sharing and pulling session data resources to multiple libraries in python selenium selenium

Selenium and BeautifulSoup: sharing and pulling session data resources to multiple libraries in python


If you simply want to pass the page source to BeautifulSoup, you can get the page source from selenium and then pass it to BeautifulSoup directly (no need of requests module).

Instead of

page = browser.current_urlr = requests.get(page)soup = BeautifulSoup(r.content, 'lxml')

you can do

page = browser.page_sourcesoup = BeautifulSoup(page, 'html.parser')