Python for the .NET developer Transcripts
Chapter: async and await in Python
Lecture: First-pass Python async
0:00 It's time to use the async and await keywords
0:02 to make this awesome.
0:04 Let me put a little caveat here.
0:06 The first pass, the first attempt that we're going to make here
0:09 is not going to be awesome for two reasons.
0:11 One, it's actually not going to add a lot of concurrency
0:14 because of the way the algorithm we're using at the bottom.
0:17 And second, the direct use of asyncio
0:20 is clumsy in Python.
0:22 It really, really should have been what they did
0:24 in C# and .NET.
0:26 They should've just said, you know what, that's good
0:28 we're going to copy that exactly.
0:30 It's a little bit clumsy, but I'm going to
0:32 show you a library that basically makes this Python version
0:35 identical to the C# version in terms of the API.
0:39 But I want to show you the native, built in version
0:41 without that library first, okay?
0:43 So what makes it clumsy?
0:44 Well, we have to import asyncio.
0:45 There's nothing clumsy about that.
0:47 But what we have to do is we have to
0:49 manually work with the scheduler, basically.
0:52 This thing called the async loops, it will say
0:54 get event loop, here.
0:57 And then we want to work with that in different places
0:59 so for example here we'll say, we want to say
1:03 run to completion, run until complete.
1:06 Want to pass this function.
1:09 In Python, you can't just start one of these
1:11 asynchronous methods.
1:12 You have to create a task out of it
1:15 and then hand it off to the event loop to be run.
1:18 C#, you can just call these methods that
1:20 return a task and do a bunch of stuff
1:22 and that's pretty cool, but here not so much.
1:25 So we're going to have to say run until complete
1:27 and at the end let's go ahead and go down here
1:30 and say, just to make sure.
1:31 Obviously this would happen at the end of the program
1:34 on it's own but we can go ahead and close out
1:35 this loop like so.
1:37 Now there's a warning saying this is not
1:38 an asynchronous method here
1:40 it's just a regular one, right?
1:41 We got None, the return type of a void.
1:44 Remember they always return None if they return nothing.
1:47 And we were looking like a generator
1:48 or something that was awaitable.
1:51 So let's go down here and work on this.
1:53 Now from here on out this basically looks
1:55 pretty much like C#'s style, okay.
1:57 So I can say this is an async
1:59 that was the wrong word, async def.
2:02 This is an asynchronous method.
2:03 And once we do that, we can then await
2:08 other asynchronous methods.
2:10 That looks like C#, right?
2:11 That's cool. So, obviously now it warns us
2:14 Hey, this is not asynchronous.
2:17 Right? So we're going to go over here
2:19 I'll have this as being an asynchronous method here
2:21 and our goal will be to convert this to be an async call.
2:25 In this little tiny regard here, Python actually
2:27 has something a little bit better than .NET, I think.
2:30 So we can have an async with.
2:32 It's like an asynchronous using block
2:34 and down here we are going to say httpx.AsyncClient
2:40 as client. We can go over here and call get
2:46 and when we do, what we get back
2:49 is actually a coroutine, not the response.
2:53 It's a coroutine of response.
2:56 So in order for this to work
2:57 we got to get the value by awaiting it.
2:59 Perfect, right? What does this do?
3:04 Why do we have to have an async thing here?
3:06 Remember this is calling basically enter and exit.
3:10 It gives this class an alternative way to work with
3:13 enter and exit, which are the implementation details
3:16 for being in a with block.
3:17 What it does is it gets an __aenter__ and __aexit__
3:20 that are asynchronous themselves, so this will
3:23 asynchronously close any open sockets and things like that.
3:26 And I'll just put a little note like that
3:28 here for you. Alright, now this should run
3:32 but it shouldn't be any better.
3:34 Let's go down here and look why.
3:36 So when this Git titles we're going to say
3:38 I'm going to go through this loop
3:39 and then I'm going to wait for one to be finished
3:42 and then go through the loop again
3:43 and wait for the next one to be finished
3:45 and go through the loop again and wait for the next one
3:47 that's the same as the synchronous version.
3:49 Other than if more stuff was happening
3:51 it could happen in parallel.
3:52 But let's just see it's working, not broken.
3:58 Hehe, look at that! T23, T24, they're rolling in.
4:01 It looks like it totally still works.
4:03 And hey, it went a lot faster. That's weird.
4:06 There's no real reason why it should go faster.
4:07 Probably just the network was faster or something.
4:10 I suspect this is not any better, but who knows
4:12 maybe there's some small improvement.
4:14 So I'm going to duplicate this
4:16 because I want to leave a copy of the old algorithm
4:18 there for you.
4:20 Now, what we did before was we came up here
4:23 we said tasks, is equal to something.
4:26 We went through and we said we want to create
4:31 some kind of tuple. If you remember that
4:33 from the C# version.
4:35 We create this list of tuples of integers and task and--
4:39 Oh my gosh, so we went through and we started the tasks
4:42 and then we created a tuple which holds
4:44 the episode number and the task.
4:47 And this started it, right?
4:48 Actually, this started that task running.
4:50 Line 31. So we're going to do something similar here.
4:55 But in Python it's a little more complicated
4:57 with this native API.
4:58 Again, this gets better, but I want to show you
5:00 the native API. So we go to our global loop
5:02 and we say, create task.
5:05 Then we give it a coroutine that's going to run.
5:08 Calling the function here doesn't actually start it
5:11 which is super annoying.
5:12 It just gets it ready to be started.
5:14 So what were we doing? We were saying get HTML.
5:19 So that's our task. And then we have episode, it's going to be N
5:24 and then into our tasks, what we did
5:26 is we added a pent here, a tuple.
5:29 So what we can put is just the episode number
5:32 and the task. Maybe I'll make that a little more explicit.
5:38 Like that. So we're going to add a tuple
5:40 this is how you do that in Python.
5:42 You just put two things separated by commas
5:44 if they're inside of a function they need extra parentheses
5:46 to indicate that there's one thing that is an argument.
5:49 We're going to have the episode number and the task.
5:51 And then down here, instead of doing this
5:53 we'll do something like, for order episode task in tasks.
6:00 We're going to do tuple unpacking here, which is beautiful.
6:03 Then we have to await the task
6:06 and this is going to be the episode number, like so.
6:10 Alright. Well, this one should start all of them.
6:14 Going to create the task.
6:15 This part starts it running, this defines it.
6:19 This starts it running on the loop.
6:21 And then we're going to add more and more and more
6:23 while there's await in, they can all run
6:25 and then we're going to make sure they're all finished.
6:27 That's the await on line 60
6:29 that basically says wait for this one to be finished.
6:32 We'll take them in order, they probably will finish out of order
6:34 but that's fine. And then once we have the string
6:36 we just go work on it in memory.
6:38 All right, let's see if this got any better.
6:39 Remember, we at best were on eight seconds here.
6:42 We had 1.6, 1.7 seconds in C# as our best option.
6:48 Let's give it a shot. They're all started.
6:51 They're all finished. Boom. Look at that!
6:53 1.03 seconds. Almost twice as fast as C#.
6:58 Incredible. It's probably just timing.
7:00 Let's run it again. No.
7:03 I've been doing this as I've been working on this class
7:04 over and over, networks have been at different modes
7:07 it's been different times of the day.
7:09 Consistently, Python is about 50% faster than C#.
7:13 I don't know why.
7:14 It seems like they're both basically doing
7:17 just a tiny little bit of work.
7:18 Possibly Beautiful Soup is much faster at parsing
7:21 HTML than HTMLAgilityPack?
7:23 Maybe the HTTP client, HTTPS versus HTTP client is better?
7:28 But either way, look at this we're crushing it.
7:30 We are completely getting a nice, stable
7:33 one second response time or time to do all of these jobs.
7:37 Man, I love it. This is great.