From the course: Web Scraping with Python

How to learn to stop worrying and love the bot - Python Tutorial

From the course: Web Scraping with Python

Start my 1-month free trial

How to learn to stop worrying and love the bot

“

- [Ryan] Are you tired of clicking on web pages one at a time and having to read them with your eyeballs? I was, until I discovered web scrapers. I built my first web scraper in order to create a database of drink recipes. Quickly, my love of potent potables was eclipsed by love of powerful programs. Soon I was web scraping full-time, building bots that pretended to be human, figuring out how websites were put together so I could break them apart, even writing scripts to beat CAPTCHAs. It was so exciting being able to use all these different disciplines of programming and computer science and funnel them often in creative ways into web scraping. Hi, I'm Ryan Mitchell and in this series, I've distilled some of the more powerful and fundamental techniques of web scraping into easy examples using Python's popular Scrapy library. So whether you're doing app testing, research, data science or just want a database of exotic drinks like I did, let's get scraping.

Contents