Async Techniques and Examples in Python Transcripts
Chapter: Built on asyncio
Lecture: Demo: unsync app for mixed-mode parallelism

Login or purchase this course to watch this video and the rest of the course contents.
0:00 Are you ready to use unsync? Let's go import it. From the module we're going to import a decorator
0:10 that we need to use. Now, let's just see a couple of things that are, let's say, annoying here. So first of all we have to create the loop
0:20 and like I said, it's fine when it's all right here but if you have a tiered application and different layers of your app need access to this loop
0:27 all the sudden that's annoying that this exists. Why can't there just be an ambient asyncio loop
0:33 that is the event loop for this thread, for this program? Boom, there is now. Second, why do we have to call create task on this?
0:42 Can't we just call this function? Like, why can't we call these functions like we can regular functions? We can, that's what we're going to do.
0:51 So now we're capturing our tasks and if we didn't actually care about the return value or waiting til everything was done we don't need to do that.
0:58 But we do want to wait til they're done. So we don't need to create the loop call, you know basically add this work into the loop
1:06 and then wait on it there. But we do want to, actually, wait before it so it's done, right. We saw how we did this with threads.
1:14 We said t.join() for t in threads. So we're going to do something similar. We'll get our value. And this we're just going to call the results.
1:27 Remember in unsync when we talked of like the beginning, this a blocking call until it's done rather that some kind of exception that it's not done.
1:35 So this line will basically make everything wait and then we can print it out. So this is step one. Basically we're removing all the junk
1:43 that we had to add for doing asyncio, right? Remove the loop, remove the adding the work and so on, and we're just going to wait
1:49 kind of as if they were threads, I guess. Now this is where it gets interesting. So we're going to go and we're going to add this unsync decorator.
2:00 Here, here. Any of these functions we want to treat this way we're going to add unsync to them. Now, this is how the rules work.
2:08 If the unsync decorator is applied to an asynchronous method it is going to run an ambient you don't have to create it but it will be there
2:17 asyncio event loop. This is all good. We don't have to do anything to make this work. This will make it run in an asyncio loop.
2:27 This one, however, it's not really an async method is it? It has the word async here, so we could mix it in into our API, but there's no await.
2:37 So let's take away that, cause it's not really meaningful to have it be that. We can still apply the unsync decorator to it and what that means is
2:44 when unsyc is applied to a regular method it will run it in the background as well but on a thread, not on a asyncio loop.
2:53 Async method, asyncio loop. Regular method on a thread. Similarly, this one async. No other comments no other values passed. Run this on asyncio
3:06 exactly as it should be. Now finally, here we have another sort of fake not useful async method, so turn it back to this.
3:15 Running it on a thread, does that help? No, we've already seen no, that does not do anything. It just makes it more work.
3:22 So unsync has a great way to deal with this. You say, this one, this operation is CPU-bound. When you say that, it's going to say
3:32 "Ah, here's a regular method. I would have run it on a thread but it's CPU-bound, so we're going to run that on multiprocessing." That's it.
3:40 Multiprocessing, best option for that one. This one, can't remember which it is, sorry. That one's async, so that's going to run asyncio.
3:50 That one does not have an async interface so we're going to run that on threads but cause it uses network it'll still release the GIL.
3:59 And that, again, asyncio. Let see if it works, let's run it and see if it does any better. Damn, look at that.
4:11 Sweet, let me make it bigger and we'll run it again. So, run it one more time. All that's going at once, all of it.
4:20 All the computing all the downloading some of the downloading on threads some of the downloading on asyncio and all the waiting on asyncio
4:28 all at once. And then we just had to wait for the computation to be done and for the downloading bits to be done. So let's look at the output.
4:36 Started out with 9.6 seconds cause there was no parallelism. That was bad. We added what we knew, what we chose what we thought
4:45 might be the best option for this particular situation. Said, "OK, so let's try asyncio." We have a lot of that in play
4:51 and that's really the most efficient if we can use it. So let's try that, we got just under 5 seconds. We threw unsync where each method
4:59 knew about its internal implementation the best way it could be run and we got this beautiful 1.3 seconds. Really, really nice.
5:09 That's unsync in a nutshell. There's not a whole lot more to it. We basically import the decorator and we just call the functions.
5:17 We don't care if they're async not async, whatever, we just call them. They all return these unfutures which we talked about at the beginning
5:25 and then you can get the results and wait on them and so on. And then we just indicate here's a CPU-bound up
5:30 regular function, here's a not CPU-bound async function. So, asyncio. Here's a regular non-async function
5:41 so threads, again, async, regular function, asyncio. And that's it. I think this is a beautiful simplification
5:50 of this whole API and it brings it all together in a really, really great way.

Talk Python's Mastodon Michael Kennedy's Mastodon