Async Techniques and Examples in Python Transcripts
Chapter: async and await with asyncio
Lecture: Demo: Make the producer-consumer async (methods)

Login or purchase this course to watch this video and the rest of the course contents.
0:00 We've got the execution of our async loop working and we've created our tasks, we've gathered them up we're waiting to run until complete.
0:09 But obviously these are standard old Python methods so what we need to do is adapt these two methods generate_data and process_data so they work.
0:19 If we try to run it now I'm not really sure what's going to happen it's got to be some kind of error, let's find out.
0:24 Run time warning, stuff is not awaited. Well they did start but when we tried to generate and when we tried to gather up the data and process it
0:35 didn't work out so well, did it? No, it did not. Yeah, right here it said you know that didn't work, we called it
0:44 it returned None and this thing said I expect a coroutine not this thing that ran and returned some value. Let's go and convert this, now there's
0:54 two things that we have to do to make our method's coroutines, asynchronous coroutines and there's two Python keywords to make this happen.
1:06 So what we have to do is we have to mark the method as an async method. So we say async like this this is a new keyword as of Python 3.5.
1:14 So now we have two async methods, and if we run them they do sort of run but notice there's no asynchronous behavior here
1:23 they're not actually asynchronous, so that's not great. Because just the fact that we say they're asynchronous
1:30 you know, Python doesn't decide how to slice them up. Our job is to take this code here and say I would like to break it into little pieces
1:40 that can run and the dividing points are where we're waiting. Well, what are we waiting on here? So PyCharm helps a little bit
1:49 and this data structure helps a lot. So there's two places we're waiting on generate_data we're waiting, clearly on line 33, we're saying time.sleep.
2:00 Now you don't want to use time.sleep this puts the whole thread and the entire asyncio loop to sleep.
2:07 There's a better way to indicate I'm done for a while I'm just going to sleep, you can keep working so we can use the asyncio's sleep, okay?
2:17 Same as time.sleep, but now there's two warnings and what are those warnings? It is PyCharm saying you have an asynchronous method
2:24 with two places you could wait you could tell the system to keep running and go do other stuff, but you're not.
2:32 So Python has a cool keyword called await and you just put that in front and these are the markers on line 30 and 33 that tells
2:40 Python to break up the code into little slices. Come in here, run until you get to this point wait and let other stuff happen here
2:48 when you're done with that and you have some time come back and do this and then wait until you're done sleeping which is, you know, .5 to 1.5 seconds
2:57 and then carry on again, okay? So we've converted this to an async method now we got to do the same thing down here.
3:06 So this is async, and this one, what we have to do is for some reason PyCharm doesn't know about the get
3:11 but if you go here you can see it's an async method. So that means that we have to await it when you await the item, it's going to block
3:21 until the return value comes out and the return value is going to be this tuple. Notice before the await, we had errors here
3:27 it was saying coroutine does not have get_item. So when you call a coroutine you have to await it to actually get the value back from it, okay?
3:37 Again, we were sleeping down here so this is going to be an asyncio.sleep and now that it's that style we have to await it as well.
3:44 So pretty simple, the two steps mark the methods as async required and then once you do that, you can use the await keyword
3:52 anywhere you would have waited. You're calling a function that is async. In this case it's kind of silly, sleep stuff
4:00 but we're going to do a much better, realistic example when we start talking to web servers and stuff, real soon. So this is sort of a toy example
4:07 we're going to get some real work done with async and await in just a moment. I think we're ready, our times are awaited and they're async
4:16 and the various places where we get and put data we're awaiting and we have the async methods here let's run this and see what we get.
4:22 Notice we don't need time anymore. Notice a very different behavior, look at this we're generating the item and then we're going
4:33 to wait on something, waiting for more data to be generated or delivered, so then we can immediately go and process it.
4:39 Look at that latency, 0, soon as an item is delivered or almost as soon, we can go and pick it up.
4:47 It took 20 seconds instead of 30, that's pretty awesome. Now we're having 0000 here because the producer is not producing as fast
4:58 as the consumer can consume. So let's take this and make one minor change here let's generate one more of those, I call that a task2
5:06 that task3, and this one we need to tell it it's expecting more data we're not dealing with cancellation and our other types of signals like that
5:14 not yet, task3, okay. So now we should be producing more data than we can consume and that should be even a little more interesting.
5:24 Notice the beginning it's kind of similar but now we sometimes process two sometimes we're processing one. The latencies are still pretty good
5:32 but like I said, we can't quite keep up, so that's okay let's see what happens at the end. Yeah, pretty good, so in the beginning we generated a couple
5:48 had to kind of do a couple double steps to catch up but in the end we were more or less able to keep up after we generated all the data
5:55 especially when one of them finished and the other was still working. Really, really cool, even generating twice as much data
6:03 it took only 22 milliseconds. I believe that would have been 50, sorry 22 seconds I believe that would have been 50 seconds
6:12 if we had done that synchronously. How cool is this? So all we had to do is to break up these functions
6:19 into parts where we're waiting on something else we're waiting on putting items into a queue we're waiting on, well, time, but like I said
6:26 that's a stand-in for database access file access, cache, network, etc. Okay so we just break them up using this await keyword
6:36 we didn't have to change our programming model at all that's the most important takeaway here. We didn't have to, all the sudden
6:42 start off a thread, use callbacks and other weird completion. It's exactly the same code that you would normally write
6:50 except for you have async and you have these await sections. So the programming model is the standard synchronous model that you know
6:57 and that's really, really nice and important. All right, that's a first introduction to asyncio. We create the loop, we create some tasks based on
7:10 some coroutines that we're going to call and then we run until they're complete.

Talk Python's Mastodon Michael Kennedy's Mastodon