Scrapy'nin sonucu belirli bir JSON biçimine nasıl dışa aktarılır?

Question

StackOverflow.com'u taramak ve atmak için Scrapy kullanıyorum. Bu so.py

import scrapy 

class StackOverflowSpider(scrapy.Spider): 
    name = 'stackoverflow' 
    start_urls = ['http://stackoverflow.com'] 

    def parse(self, response): 
     for href in response.css('.question-summary h3 a::attr(href)'): 
      full_url = response.urljoin(href.extract()) 
      yield scrapy.Request(full_url, callback=self.parse_question) 

    def parse_question(self, response): 
     yield { 
      'link': response.url, 
     } 

Beklenen sonuç: so.json(geçerli JSON biçimi)

[ 
    "http://stackoverflow.com/questions/36421917/exponential-number-in-custom-number-format-of-excel", 
    "http://stackoverflow.com/questions/36421343/can-not-install-requirements-txt", 
    "http://stackoverflow.com/questions/36418815/difference-between-two-approaches-to-pass-parameters-to-web-server", 
    "http://stackoverflow.com/questions/36421743/sharing-an-oracle-database-connection-between-simultaneous-celery-tasks", 
    "http://stackoverflow.com/questions/36421941/jquery-add-css-style", 
] 

Sonra çalıştırın:

scrapy runspider so.py -o so.json 

sonucu değil beklendiği gibi. Buraya sıkıştım.

Answer 1

FEED_FORMAT=jsonlines ayarını kullanmayı deneyin. Eğer

[ 
    "https://stackoverflow.com/questions/36421917/exponential-number-in-custom-number-format-of-excel", 
    "https://stackoverflow.com/questions/36421343/can-not-install-requirements-txt", 
    "https://stackoverflow.com/questions/36418815/difference-between-two-approaches-to-pass-parameters-to-web-server", 
    "https://stackoverflow.com/questions/36421743/sharing-an-oracle-database-connection-between-simultaneous-celery-tasks", 
    "https://stackoverflow.com/questions/36421941/jquery-add-css-style", 
] 

Kendi ItemExporter yazmalısınız almak istiyorsanız

scrapy runspider so.py -o so.json --set FEED_FORMAT=jsonlines 

, see this question