From the course: Web Scraping with Python

Unlock the full course today

Join today to access over 22,600 courses taught by industry experts or purchase this course individually.

Challenge: Scraping all data on a page

Challenge: Scraping all data on a page - Python Tutorial

From the course: Web Scraping with Python

Start my 1-month free trial

Challenge: Scraping all data on a page

(upbeat music) - [Instructor] In the last video, we very quickly touched on how to work with the scrapy response objects in a function and extract the data that we want from them. So remember, there are two main ways to do this, CSS selectors and XPath selectors. Although CSS notation may be tempting to use at first, especially if you have a lot of experience with front-end web development and are very used to using CSS, I would caution against it. CSS selectors are far less powerful than XPath. Although some pseudo classes have been added to work around some of the limitations, you'll likely run into long-term problems if you stick with CSS and completely ignore XPath. So we'll also be using XPath throughout these videos. The XPath syntax can seem a little convoluted at first, but it's actually extremely flexible and powerful, and you can do just about anything with it. It's like the regular expressions for XML or…

Contents