Trying to use Python and Selenium to scroll and scrape a webpage iteratively Trying to use Python and Selenium to scroll and scrape a webpage iteratively selenium selenium

Trying to use Python and Selenium to scroll and scrape a webpage iteratively


From what I see on the screenshot, it looks like you need to iteratively scroll into view of the last row in the table - the last element with ag-row class:

import time   while True:    rows = driver.find_elements_by_css_selector("tr.ag-row")    driver.execute_script("arguments[0].scrollIntoView();", rows[-1])    time.sleep(1)    # TODO: collect the rows

You would also need to figure out the loop exit condition.