From the course: Web Scraping with Python
Join today to access over 22,600 courses taught by industry experts or purchase this course individually.
Browser automation with Selenium - Python Tutorial
From the course: Web Scraping with Python
Browser automation with Selenium
- [Instructor] Remember in chapter three, when we looked for Dunkin Donuts locations, but they were tricky to find because they were loaded dynamically on the page? Well, there are multiple approaches to this kind of a problem as it turns out. Personally, my favorite is just to look for the hidden API requests, just like we did in chapter three, find the original source of the data and then use that API request in your scraper. But now we're going to do something which is not quite as slick or elegant, but way more impressive looking. It's actually a really useful tool to use in a variety of situations. So that's right, we're going to bring out the big guns, the MOAB, that is the Mother Of All Bots, of course. And we're talking browser automation. To do this, we're going to need to grab a couple of things. So first, install a library called scrapy-selenium. This is a Python package that installs a browser automation tool…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.