Scrapy FormRequest sending JSON Scrapy FormRequest sending JSON json json

Scrapy FormRequest sending JSON


FormRequest is for simulating an HTML form (e.g. application/x-www-form-urlencoded). It sounds like you are simply wanting to POST data with your Request. Since you mention a content type of 'application/json' you probably want to do something like this:

request = Request( url, method='POST',                    body=json.dumps(my_data),                    headers={'Content-Type':'application/json'} )


from scrapy.http import JsonRequestclass TestSpider(scrapy.Spider):    def start_requests(self):        data = {            'name1': 'value1',            'name2': 'value2',        }        yield JsonRequest(url='http://www.example.com/post/action', data=data)

see more: docs.scrapy.org#jsonrequest


You could just use the below snippet for getting response:

res =requests.post(url, json=data) print(res.text)