Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Demo: Executor app introduction
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's look at a new program. Here we are in our execution pool section of our GitHub repo and I have created a new program and I have it all set up.
0:08
It requires two particular packages requests and Beautiful Soup. What we want to do is we're going to use either threading
0:17
or multiprocessing to implement this. And, what we'd like to do is make this as simple as possible to switch between those.
0:24
So we're not tied deeply, deeply to the API. It's more of a decision of we're using the API and now which mode do we want it to work in.
0:32
So, we're going to look at the synchronous version and then convert it to one we can toggle this mode. Alright, so we've got a set of URLs here
0:41
and the goal is we're going to go to those URLs and get the title that comes in. Actually, we're going to get the H1. So, whatever the visible title is
0:49
not the head part of the HTML. So we've got this function called get_title() And see we're just straightforward
0:56
looping over those and we come down here. It's using requests, now we could use asyncio but the point is we're going to get to that later.
1:04
There's actually a second step farther down in this course where we unify that as well but right now what we're talking about only applies to the older
1:13
parallel techniques, so threading and multiprocessing. Alright, so because of that we're using requests and not aiohttp. We're going to
1:20
suck down this HTML, feed it to Beautiful Soup and tell Beautiful Soup to go through the DOM and find the thing that's called H1.
1:27
There's a little bit of weirdness around where the title is set for some of the pages and then it just gives back the text of that
1:33
H1 and we call that the title. Let's run it and see it in action. It's getting the title from Python.fm which is Talk Python to Me, and Pythonbytes.fm
1:43
which is The Python Bytes Podcast. Google has none, I think it's just an image or something funky like that. Real Python is Real Python Tutorials
1:51
and training.talkpython.fm is Talk Python Training. Okay, so that you can see is sort of running kind of quick but this is the perfect example of
2:00
where something might be really nice to apply parallelism. Like we saw in the asyncio chapter we're really just waiting on these web requests.
2:09
We're not actually doing tons of work. So this is a good candidate for threading but it also may be a good candidate for multiprocessing.
2:16
This is the app that we're going to work with and we're going to convert it to use one or the other of those two parallel techniques.