Issue with loop in scrapy+selenium+phantomjs
If I correctly understood what you want to do, I think this one could help you.
The problem is that start_urls are not processed in order. They are passed to start_requests method and returned with a downloaded response to parse method. This is asynchronous.
Maybe this helps
#Do your thingstart_urls = [open('bobo.txt').readlines()[0].strip()]other_urls = [l.strip() for l in open('bobo.txt').readlines()[1:]]other_urls.reverse()#Do your thingdef parse(self, response): #Do your thing if len(self.other_urls) != 0 url = self.other_urls.pop() yield Request(url=url, callback=self.parse)