#100DaysOfCode in Python Transcripts
Chapter: Days 46-48: Web Scraping with BeautifulSoup4
Lecture: Your 3 day overview
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
This is the readme file for beautifulsoup4, for Day 46 to 48 on web stripping. Now, day N the first day you're going to be
0:12
working on this course, I would like you be watching the video on setting up the environment, getting a quick overview of beautifulsoup4.
0:22
This is if you have no familiarity on what it is and how it works, and then build your first Beautiful Soup 4 scrapper.
0:31
It's actually not too much work, but there is a bit of theory there, with the overview, and you should be able to get it up and running,
0:39
and then give it a crack yourself, okay? Watch the videos first, because if you're not familiar
0:46
with it, it does help to watch it start to finish, okay? Pull your first site, use the example site in the video, or if you really want to
0:55
challenge yourself, grab another one. Day 2, what I'd like you to do is watch this video on requests best practice, okay?
1:05
This is covering a little thing that people tend to do with requests that is actually the wrong way to do it,
1:12
and we discuss the best practice for actually doing it, I won't give it away now. Then, what I'd like you to do is follow
1:20
along with this video, detailed Beautiful Soup 4 scrapping and searching. This will actually go through how to do some targeted searching, so to speak,
1:32
of the data that you pull down and scrape, okay? It can be a bit tricky and a bit frustrating to find exactly what you want,
1:40
but stick with it and you'll get there in the end. And Day 3, as usual, it's your turn. So, you've figured out how to scrape a website,
1:49
you can pull the data that you want, so now I'd like you to actually do something with it, okay? So store it database, display it in something
1:58
like a Flask app or a GUI, automate it by emailing it, do whatever you can think of, right? So, come up with something and do that.
2:08
If you can't think of anything, you could try this one. I've added an extra option here for you to try, which is to find a site that looks complex.
2:18
Think of something that maybe has Flash, or whatever other animations on the website. Pinpoint a data sample, so just something on
2:26
the website you think that could be interesting, and then see if you can extract it using Beautiful Soup 4, okay?
2:35
So that's it, give anything like that a try. Day 3 is your freestyle, free-for-all, do whatever you want, and just have a good play.
2:43
And, move on to the videos and get started.