Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Demo: Executor app introduction
0:00 Let's look at a new program.
0:01 Here we are in our execution pool
0:03 section of our GitHub repo and I have
0:05 created a new program and I have it all set up.
0:07 It requires two particular packages
0:10 requests and Beautiful Soup. What we want to do
0:13 is we're going to use either threading
0:16 or multiprocessing to implement this.
0:18 And, what we'd like to do is make this
0:20 as simple as possible to switch between those.
0:23 So we're not tied deeply, deeply to the API.
0:26 It's more of a decision of we're using the API
0:29 and now which mode do we want it to work in.
0:31 So, we're going to look at the synchronous version
0:33 and then convert it to one we can toggle this mode.
0:37 Alright, so we've got a set of URLs here
0:40 and the goal is we're going to go to those
0:41 URLs and get the title that comes in.
0:44 Actually, we're going to get the H1.
0:45 So, whatever the visible title is
0:48 not the head part of the HTML.
0:51 So we've got this function called get_title()
0:53 And see we're just straightforward
0:55 looping over those and we come down here.
0:57 It's using requests, now we could use
0:59 asyncio but the point is we're going to get to that later.
1:03 There's actually a second step farther down in this course
1:06 where we unify that as well but right now
1:09 what we're talking about only applies to the older
1:12 parallel techniques, so threading and multiprocessing.
1:15 Alright, so because of that we're using
1:17 requests and not aiohttp. We're going to
1:19 suck down this HTML, feed it to Beautiful Soup
1:22 and tell Beautiful Soup to go through the DOM
1:24 and find the thing that's called H1.
1:26 There's a little bit of weirdness around
1:28 where the title is set for some of the pages
1:30 and then it just gives back the text of that
1:32 H1 and we call that the title.
1:34 Let's run it and see it in action.
1:37 It's getting the title from python.fm
1:39 which is Talk Python to Me, and pythonbytes.fm
1:42 which is The Python Bytes Podcast.
1:44 Google has none, I think it's just an
1:46 image or something funky like that.
1:48 Real Python is Real Python Tutorials
1:50 and training.talkpython.fm is Talk Python Training.
1:54 Okay, so that you can see is sort of running
1:57 kind of quick but this is the perfect example of
1:59 where something might be really nice to apply parallelism.
2:02 Like we saw in the asyncio chapter
2:05 we're really just waiting on these web requests.
2:08 We're not actually doing tons of work.
2:10 So this is a good candidate for threading
2:12 but it also may be a good candidate for multiprocessing.
2:15 This is the app that we're going
2:16 to work with and we're going to convert it
2:18 to use one or the other of those two parallel techniques.