TypeError when putting scraped data from scrapy into elasticsearch
As you can see in the error message: Error processing {'link': [u'http://www.meetup.com/Search-Meetup-Karlsruhe/events/221907250/'], 'title': [u'Alles rund um Elasticsearch']}
your item's link
and title
fields are lists (the square brackets around the values indicate this).
This is because of your extraction in Scrapy. You did not post it with your question but you should use response.xpath().extract()[0]
to get the first result of the list. Naturally in this case you should prepare to encounter empty result sets to avoid index-errors.
Update
For the situation where you do not extract anything you could prepare with the following:
linkSelection = response.xpath().extract()item['link'] = linkSelection[0] if linkSelection else ""
Or something alike depending on your data and fields. Perhaps None
could be valid too if the list is empty.
The basic idea is to split up XPath extraction and list-item selection. And you should select an item from the list if it contains the required elements.