Python for .NET Developers Transcripts
Chapter: async and await in Python
Lecture: First-pass Python async
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
It's time to use the async and await keywords to make this awesome. Let me put a little caveat here.
0:07
The first pass, the first attempt that we're going to make here is not going to be awesome for two reasons.
0:12
One, it's actually not going to add a lot of concurrency because of the way the algorithm we're using at the bottom.
0:18
And second, the direct use of asyncio is clumsy in Python. It really, really should have been what they did in C# and .NET.
0:27
They should've just said, you know what, that's good we're going to copy that exactly. It's a little bit clumsy, but I'm going to
0:33
show you a library that basically makes this Python version identical to the C# version in terms of the API.
0:40
But I want to show you the native, built in version without that library first, okay? So what makes it clumsy? Well, we have to import asyncio.
0:46
There's nothing clumsy about that. But what we have to do is we have to manually work with the scheduler, basically.
0:53
This thing called the async loops, it will say get event loop, here. And then we want to work with that in different places
1:00
so for example here we'll say, we want to say run to completion, run until complete. Want to pass this function.
1:10
In Python, you can't just start one of these asynchronous methods. You have to create a task out of it
1:16
and then hand it off to the event loop to be run. C#, you can just call these methods that return a task and do a bunch of stuff
1:23
and that's pretty cool, but here not so much. So we're going to have to say run until complete and at the end let's go ahead and go down here
1:31
and say, just to make sure. Obviously this would happen at the end of the program on it's own but we can go ahead and close out this loop like so.
1:38
Now there's a warning saying this is not an asynchronous method here it's just a regular one, right? We got None, the return type of a void.
1:45
Remember they always return None if they return nothing. And we were looking like a generator or something that was awaitable.
1:52
So let's go down here and work on this. Now from here on out this basically looks pretty much like C#'s style, okay. So I can say this is an async
2:00
that was the wrong word, async def. This is an asynchronous method. And once we do that, we can then await other asynchronous methods.
2:11
That looks like C#, right? That's cool. So, obviously now it warns us Hey, this is not asynchronous. Right? So we're going to go over here
2:20
I'll have this as being an asynchronous method here and our goal will be to convert this to be an async call.
2:26
In this little tiny regard here, Python actually has something a little bit better than .NET, I think. So we can have an async with.
2:33
It's like an asynchronous using block and down here we are going to say httpx.AsyncClient as client. We can go over here and call get
2:47
and when we do, what we get back is actually a coroutine, not the response. It's a coroutine of response. So in order for this to work
2:58
we got to get the value by awaiting it. Perfect, right? What does this do? Why do we have to have an async thing here?
3:07
Remember this is calling basically enter and exit. It gives this class an alternative way to work with
3:14
enter and exit, which are the implementation details for being in a with block. What it does is it gets an __aenter__ and __aexit__
3:21
that are asynchronous themselves, so this will asynchronously close any open sockets and things like that. And I'll just put a little note like that
3:29
here for you. Alright, now this should run but it shouldn't be any better. Let's go down here and look why. So when this Git titles we're going to say
3:39
I'm going to go through this loop and then I'm going to wait for one to be finished and then go through the loop again
3:44
and wait for the next one to be finished and go through the loop again and wait for the next one that's the same as the synchronous version.
3:50
Other than if more stuff was happening it could happen in parallel. But let's just see it's working, not broken.
3:59
Hehe, look at that! T23, T24, they're rolling in. It looks like it totally still works. And hey, it went a lot faster. That's weird.
4:07
There's no real reason why it should go faster. Probably just the network was faster or something. I suspect this is not any better, but who knows
4:13
maybe there's some small improvement. So I'm going to duplicate this because I want to leave a copy of the old algorithm there for you.
4:21
Now, what we did before was we came up here we said tasks, is equal to something. We went through and we said we want to create
4:32
some kind of tuple. If you remember that from the C# version. We create this list of tuples of integers and task and--
4:40
Oh my gosh, so we went through and we started the tasks and then we created a tuple which holds the episode number and the task.
4:48
And this started it, right? Actually, this started that task running. Line 31. So we're going to do something similar here.
4:56
But in Python it's a little more complicated with this native API. Again, this gets better, but I want to show you
5:01
the native API. So we go to our global loop and we say, create task. Then we give it a coroutine that's going to run.
5:09
Calling the function here doesn't actually start it which is super annoying. It just gets it ready to be started.
5:15
So what were we doing? We were saying get HTML. So that's our task. And then we have episode, it's going to be N and then into our tasks, what we did
5:27
is we added a pent here, a tuple. So what we can put is just the episode number and the task. Maybe I'll make that a little more explicit.
5:39
Like that. So we're going to add a tuple this is how you do that in Python. You just put two things separated by commas
5:45
if they're inside of a function they need extra parentheses to indicate that there's one thing that is an argument.
5:50
We're going to have the episode number and the task. And then down here, instead of doing this
5:54
we'll do something like, for order episode task in tasks. We're going to do tuple unpacking here, which is beautiful. Then we have to await the task
6:07
and this is going to be the episode number, like so. Alright. Well, this one should start all of them. Going to create the task.
6:16
This part starts it running, this defines it. This starts it running on the loop. And then we're going to add more and more and more
6:24
while there's await in, they can all run and then we're going to make sure they're all finished. That's the await on line 60
6:30
that basically says wait for this one to be finished. We'll take them in order, they probably will finish out of order
6:35
but that's fine. And then once we have the string we just go work on it in memory. All right, let's see if this got any better.
6:40
Remember, we at best were on eight seconds here. We had 1.6, 1.7 seconds in C# as our best option. Let's give it a shot. They're all started.
6:52
They're all finished. Boom. Look at that! 1.03 seconds. Almost twice as fast as C#. Incredible. It's probably just timing. Let's run it again. No.
7:04
I've been doing this as I've been working on this class over and over, networks have been at different modes it's been different times of the day.
7:10
Consistently, Python is about 50% faster than C#. I don't know why. It seems like they're both basically doing just a tiny little bit of work.
7:19
Possibly Beautiful Soup is much faster at parsing HTML than HTMLAgilityPack? Maybe the HTTP client, HTTPS versus HTTP client is better?
7:29
But either way, look at this we're crushing it. We are completely getting a nice, stable one second response time or time to do all of these jobs.
7:38
Man, I love it. This is great.