Scrapy FormRequest sending JSON
FormRequest is for simulating an HTML form (e.g. application/x-www-form-urlencoded). It sounds like you are simply wanting to POST data with your Request. Since you mention a content type of 'application/json' you probably want to do something like this:
request = Request( url, method='POST', body=json.dumps(my_data), headers={'Content-Type':'application/json'} )
from scrapy.http import JsonRequestclass TestSpider(scrapy.Spider): def start_requests(self): data = { 'name1': 'value1', 'name2': 'value2', } yield JsonRequest(url='http://www.example.com/post/action', data=data)
see more: docs.scrapy.org#jsonrequest