Selenium and BeautifulSoup: sharing and pulling session data resources to multiple libraries in python
If you simply want to pass the page source to BeautifulSoup
, you can get the page source from selenium
and then pass it to BeautifulSoup
directly (no need of requests
module).
Instead of
page = browser.current_urlr = requests.get(page)soup = BeautifulSoup(r.content, 'lxml')
you can do
page = browser.page_sourcesoup = BeautifulSoup(page, 'html.parser')