Iterating over links in selenium with scrapy
You can take another approach: call browser.get() for every href of a link in the loop:
links = self.browser.find_elements_by_xpath('//ol[@class="listing"]/li/h4/a')for link in links: link = link.get_attribute('href') self.browser.get(link) # crawlIf a link is relative you would need to join it with the http://www.somepage.com.