Scrapy Pipeline To Parse

I made a pipeline to put scrapy data to my Parse Backend PARSE = 'api.parse.com' PORT = 443 However, I can't find the right way to post the data in Parse. Because everytime it crea

Solution 1:

Looks like you have a set inside item['data'], which isn't accepted on JSON.

You need to change that field back to list before trying to make it JSON acceptable.

Solution 2:

I found the solution

class Newscrawlbotv01Pipeline(object):
def process_item(self, item, spider):
    for data in item:
        if not data:
            raise DropItem("Missing data!")
    connection = httplib.HTTPSConnection(
        settings['PARSE'],
        settings['PORT']
    )

    connection.connect()
    connection.request('POST', '/1/classes/Articles', json.dumps(dict(item)), {
   "X-Parse-Application-Id": "WW",
   "X-Parse-REST-API-Key": "WW",
   "Content-Type": "application/json"
 })
    log.msg("Question added to PARSE !", level=log.DEBUG, spider=spider)
    return item
    #self.collection.update({'url': item['url']}, dict(item), upsert=True)