I Am Getting An Attributeerror: 'htmlresponse' Object Has No Attribute 'xpath' In Scrapy

I am newbie for scrapy and I am using Scrapy 0.14.4. I just want to print title and link as per following example. Here is my spider: from scrapy.spider import BaseSpider class Xx

Solution 1:

The problem is that you are using an old version of Scrapy where selectors were not included in the response objects. To verify this look at the relevant documentation: http://doc.scrapy.org/en/0.14/topics/request-response.html

To solve your problem wrap the response into a selector and then you can use the xpath function on the selector:

from scrapy.selector import HtmlXPathSelector 
defparse(self, response):
    hxs = HtmlXPathSelector(response)
    for sel in hxs.select("//div[@id='job_listings']/a"):