How to scroll down in an instagram pop-up frame with Selenium
the element you're looking is //div[@class='isgrP']
and Keys.PAGE_DOWN
is not work for scrollable div.
Your variable fList
hold old value, you need to find again the elements after scroll.
#find all li elements in listfBody = driver.find_element_by_xpath("//div[@class='isgrP']")scroll = 0while scroll < 5: # scroll 5 times driver.execute_script('arguments[0].scrollTop = arguments[0].scrollTop + arguments[0].offsetHeight;', fBody) time.sleep(tryTime) scroll += 1fList = driver.find_elements_by_xpath("//div[@class='isgrP']//li")print("fList len is {}".format(len(fList)))print("ended")#driver.quit()
The above code works fine if you add iteration (for) with rangefor i in range(1, 4): try:
#find all li elements in list fBody = self.driver.find_element_by_xpath("//div[@class='isgrP']") scroll = 0 while scroll < 5: # scroll 5 times self.driver.execute_script('arguments[0].scrollTop = arguments[0].scrollTop + arguments[0].offsetHeight;', fBody) time.sleep(2) scroll += 1 fList = self.driver.find_elements_by_xpath("//div[@class='isgrP']//li") print("fList len is {}".format(len(fList))) except Exception as e: print(e, "canot scrol") try: #get tags with a hrefs_in_view = self.driver.find_elements_by_tag_name('a') # finding relevant hrefs hrefs_in_view = [elem.get_attribute('title') for elem in hrefs_in_view] [pic_hrefs.append(title) for title in hrefs_in_view if title not in pic_hrefs] print("Check: pic href length " + str(len(pic_hrefs))) except Exception as tag: print(tag, "can not find tag")
So, the for loop makes it to possible scrol even if the while loop miss