Selenium web scraping in python cant read .text of elements Selenium web scraping in python cant read .text of elements selenium selenium

Selenium web scraping in python cant read .text of elements


You should improve the logic of your code. Note, that you cannot get text of elements from the first page after redirection to next page- you need to get text before clicking "Next" button.

Try to use below code instead:

from selenium import webdriverfrom selenium.common.exceptions import WebDriverExceptionimport timeurl = 'https://www.verizonwireless.com/smartphones/samsung-galaxy-s7/'browser = webdriver.Chrome()browser.get(url)reviews = []xp = '//a[span[@class="bv-content-btn-pages-next"]]'# read first ten pages of reviews ==> for i in range(10):    for review in browser.find_elements_by_xpath('//div[@class="bv-content-summary-body-text"]/p'):        reviews.append(review.text)    try:        next = browser.find_element_by_xpath(xp)        next.location_once_scrolled_into_view        time.sleep(0.5) # To wait until scrolled down to "Next" button        next.click()        time.sleep(2) # To wait for page "autoscrolling" to first review + until modal window dissapeared    except WebDriverException:        print("error clicking")for review in reviews:    print(review)