#100DaysOfWeb in Python Transcripts
Chapter: Days 73-76: Web Scraping
Lecture: Prepare to scrape!
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
We'll do a quick, simple setup video here quick overview of what we're going to do. Python -m, venv venv inside a new directory six-webscraping.
0:13
This is within our 100 Days of Web projects folder that we've been using to date. Now, once you've got your virtual environment installed
0:21
go ahead and launch it. Get it going, and then we're going to pip install the two modules we're going to use.
0:29
We're going to use bs4, which is Beautiful Soup 4 and then we're going to install the requests library as well. These two are required to web scrape.
0:41
Requests will pull down the website and bs4 will parse it for us. Right, there we go. And next up, before we actually do anything
0:52
let's take a look at the website I'd like you to scrape for this day. We're going to actually scrape the training.talkpython.fm course's webpage.
1:02
So, what the intent is, is to show you this is what the website looks like on the front end and then we're going to scrape it
1:12
pull it down, and pull out specific bits from this page. If you right click on your page and click on a view page source
1:21
it will open up a new tab and you'll be able to see all of the HTML. Now, we're really lucky Mike has made this a really simple website to scrape.
1:31
It's nice and neat, nice and tidy. I really like it, that's why I've chosen it to teach you bs4. Open up a view source.
1:39
Also of use to you is to click on inspect when you right click on the page and that will open up a little inspect window here
1:49
where, when you mouse over the code your HTML, it actually highlights it on the screen so if we hover over the div here
1:59
we can see that that is actually containing our h1 tag and that says online video courses at Talk Python Training.
2:07
That's where I'm going to leave this video. Get yourself your browser set up. I'm using Chrome, obviously, but you can do this
2:14
with any other browser. Firefox is just fine. So get yourself set up, get the website up get your view source up
2:21
and get your virtual environment installed with your two libraries. Move on to the next video.