Scrapy -- When calling scrapy from script, how to return a dictionary to the process immediately, instead of export to json file Scrapy -- When calling scrapy from script, how to return a dictionary to the process immediately, instead of export to json file json json

Scrapy -- When calling scrapy from script, how to return a dictionary to the process immediately, instead of export to json file


Feed exports are meant to serialize the data you've scraped (see feed export documentation). What you are trying to do doesn't involve serialization.

What you want to do instead is create a pipeline. Scrapy will pass scraped Items to the pipeline. They are dictionaries, and you can do whatever you want with them.