Async Techniques and Examples in Python Transcripts
Chapter: async and await with asyncio
Lecture: Concept: async web scraping

Login or purchase this course to watch this video and the rest of the course contents.
0:00 We've seen just how powerful asynchronous web requests can be. After all, what are you doing when you're making a web request?
0:08 You're waiting on the server. So, you can wait on a whole lot of servers all at the same time, and get a ton done concurrently.
0:15 So, let's review some of the concepts around it. Well, of course, we're going to have an asynchronous method this get_HTML method.
0:23 If it's not async, we can't do this. But assuming we're within an asynchronous method what we need to do is we're going to need to find
0:29 these async with blocks for the aiohttp library. So, we're going to create a client session and then from that session we're going to
0:37 asynchronously do a get request that'll give us a response. That one's the one that probably takes the longest. Want to do this raise_for_status
0:46 just to make sure that we got to 200 or some form of success code. And, by the way, that does not come up in PyCharm
0:54 when you say resp.raise_for_status() doesn't come up but it is there and it works. And then finally, we want to read
1:01 the content from the network stream. So not just start the response but actually get all the content back.
1:07 So, we'll say resp.text and that's also a coroutine so we're going to await it as well. Convert it from a coroutine into a string and send that back.
1:17 So, here's how we do asynchronous web processing web request, with aiohttp and asyncio.


Talk Python's Mastodon Michael Kennedy's Mastodon