Python for .NET Developers Transcripts
Chapter: async and await in Python
Lecture: More threading with unsync
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Remember this picture? I said you can use asyncio to weight on I/O type of completion. You can use threads if somehow those threads are being
0:09
the GIL is somehow being released like with the thread.sleep or we're talking about some kind of network thing.
0:14
But if you wanted true computational and parallelism you got to use multi processing and whoo! You're like, jeez why is this so complicated?
0:20
Why can't I just use a task for all of them? That's where unsync comes in! So, we saw unsync already simplified the
0:26
asyncio API. What it also does is unify asyncio threads and multi processing all behind async await. And it's actually super super cool.
0:36
So let's go look at that. Now, there's not really any parallelism that's super interesting here but let's just go and throw in some stuff
0:45
that we can do. So, here's the rules about unsync that I haven't talked about yet, because it actually does more than just asyncio.
0:52
If I apply this decorator to an asynchronous function it will run on this implicit, behind-the-scenes
0:59
AIO event loop, that it manages on a background thread. Okay? That's pretty awesome! So, we just put it on asynchronous methods
1:07
and it does all the loop juggling for us. But it does more. If I did this and I put it here this will let's go like this, maybe make a little comment.
1:17
So this will run this function on a background thread. Yes, so if we say unsync on a non-asynchronous method, it means it cannot run on an event loop.
1:29
Unsync will go well, you still want to do something asynchronous with it, so let's run it on a background thread. And see this get title from HTML?
1:35
Now, here now we have to await this as well. As if it's going to complain because the style is let's just not worry about it. It's still going to work
1:45
like I said, there's some weird type detection stuff between the decorators and PyCharm, let's just see it works. Yep! Oof, took forever that time!
1:56
Let's try it again. You know, that's probably a lot slower, isn't it? Oh, no no, it's probably just timing. Look, it's much faster.
2:02
It does seem consistently a little bit slower. Ah, I don't know. Networks! If we put it like this, it's going to run
2:09
on a separate thread, and then it becomes something we can await exactly like if it were asyncio. Right, that's super cool already.
2:17
But remember, this the way this works it doesn't make any sense to run this on a background thread because the GIL is still at play.
2:25
Even if unsync is trying to manage the API for us. It's still not really going to run more than one instruction at a time. So it doesn't help us.
2:33
What we have to do in Python is run in a sub process. Now it's pretty easy, basically go to this multi processing API and you give it a function
2:41
you say run that in another process and give me the return value. Though it's not, like, a lot of work that you have to do
2:46
but it is a very different mode of execution. And that's really, at the moment, what's required to do computational parallels in Python.
2:53
Well, unsync has a cool thought on that as well. Unsync says, you know, we can go over here and say @unsync(cpu_bound=True). And if we say that
3:05
then unsync knows, hey this is a computational thing Python doesn't do that well with threads. So let's do that with multi processing.
3:12
Just changing that to that converts from using threads to using multiprocessing. And I guess I can put a little comment here as well.
3:22
Here we go. So, this is the more appropriate one. It might be slower, there is the start up of the processing and this execution is really fast.
3:30
So, let's run and see if it's any better. I might just comment it out and say we're not going to do this.
3:34
But, it's incredible that when these approaches are relevant for the type of execution, the algorithm you have
3:40
that all you have to do is put an unsync decorator on a regular function, and it becomes something that runs on a thread and you can await it
3:47
or CPU bound equals true and it's something that runs in another process and you still can await it down here like this. Nevermind that warning
3:55
that's Pycharm badly guessing at what it should do. Alright, again done. 1.10 seconds. Try again. 1.12. It's not really making it better. 1.09
4:07
It's not making it much worse. I think this is mostly just the network discrepancies that we're seeing. Right, well, I guess I'll leave it like this
4:15
like if this were me and these numbers were coming out that similar I would just just call this the regular function. I wouldn't go to that trouble.
4:21
But, there are many many examples where this execution takes a lot longer. And this would make a big difference. Like, lets go down here and say 'time'
4:29
import 'time' at the top. Yeah, we can trick it here. Sleep, this is like, this is like the sleep thread dot sleep and dot net.
4:37
Let's say it takes an extra hundred oh, that's seconds Let's say it takes a hundred milliseconds or two hundred milliseconds to do this.
4:45
And first, let run it directly. Down here You're not going to await it. That's pretty, pretty tall isn't it? Can't await it if we're going to do that.
4:55
So let's just see how long it takes like that. There you can see we got the the data back right away and it started processing it.
5:06
But it could only process them one at a time. So that was two hundred milliseconds per time. Right, as that number gets better, or bigger
5:12
it gets worse. So let's go over here and say we're going to try to use multiprocessing to do that. Does it make it better? it's actually
5:24
I just have to await down here, remember? I didn't get I got a coroutine chain back and then I tried to make it green.
5:31
Apparently you can't make coroutines green. So weird! Any better? Ah, a little bit! Three, Four, yeah I'm not sure it's any better.
5:46
I don't know how much parallels I'm we're getting out of our multiprocessing there. Anyway, I'm going to comment these out
5:52
and put it back just run it normal. But, this unsync has a really cool API. Oh yeah, let's take away this as well.
6:01
Put it back. And it goes beyond just working with asyncio. It allows us to take regular non asynchronous methods
6:09
and convert them to either threaded mode or multiprocessing mode. Which is pretty slick. If you come from a C# background
6:16
you definitely should check out unsync. It really is basically an adaptation of Python's model over to what you would get from
6:23
the task parallel library and C#.