Skip to content Skip to sidebar Skip to footer

If Statement Within Scrapy Item Declaration

I'm using scrapy to build a spider to monitor prices on a website. The website isn't consistent in how it displays it's prices. For it's standard price, it always uses the same CSS

Solution 1:

I got it to work. Here's my code:

    item = ScraperItem()

    item['model'] = extract_with_css('.product-id::text')
    item['link'] = extract_with_css('head meta[property="og:url"]::attr(content)')
    item['price'] = extract_with_css('span.list-price:last-child::text')

    if response.css('span.price-num:last-child::text'):
        item['promo_price'] = extract_with_css('span.price-num:last-child::text')
    else:
        item['promo_price'] = extract_with_css('.product-highlight-label::text')

    yield item

Post a Comment for "If Statement Within Scrapy Item Declaration"