Donate. I desperately need donations to survive due to my health

Get paid by answering surveys Click here

Click here to donate

Remote/Work from Home jobs

MessageformatchangedinRabbitMQ

on

I'm using scrapy-rabbitmq-link to get message from RabbitMQ.

The message is RabbitMQ is

{"title": "Thousands in Georgia protest against presidential election results", "url": "https://www.reuters.com/article/georgia-protest/thousands-in-georgia-protest-against-presidential-election-results-idUSL8N1Y70HE", "timestamp": "2018-12-02T15:16:02+00:00", "tickers": []}

After the crawler working several minutes, some of the message format changed to https://www.reuters.com/article/georgia-protest/thousands-in-georgia-protest-against-presidential-election-results-idUSL8N1Y70HE without the original format. Most of the message can be processed successfully. It seems if something is wrong, the url send back to the queue but lost the original message format.

Please advice!

Comments

Post a Comment