#100DaysOfWeb in Python Transcripts
Chapter: Days 21-24: Async Flask APIs with Quart
Lecture: Synchronous execution demo
0:00 All right, enough talk, let's write some code.
0:02 So over here in our demo repository
0:05 we can see the days, and this is the quart-async section.
0:09 So what we're going to do is we're going to create a simple
0:12 project that we can work with
0:14 and we're going to write some regular synchronous code
0:16 and then we're going to convert to async
0:18 and see how things get better.
0:21 All, right so first, let me just make a folder here.
0:25 And I'll just call this python_async.
0:28 Now this part's not technically necessary
0:30 but I do want to use one external package
0:32 and because of that, I'm going to create a virtual environment
0:34 that we can use here.
0:47 So we're just going to use python3
0:48 to create a virtual environment.
0:52 And then I want to go and open this in PyCharm.
0:54 Use whatever editor you want, I'm going to use PyCharm.
0:58 All right, with that open, let's create
1:00 a file that we're going to run.
1:02 We're going to have a synchronous version
1:03 and a asynchronous version.
1:09 I'll just call that synch_program
1:10 not super exciting, but here we go.
1:13 Now, for this one, I'm going to start with some code
1:14 that I'm going to paste in here
1:16 and we're just going to talk through it because
1:18 it's a synchronous program, you guys
1:20 already know how to write this stuff.
1:21 I want to have a really clear foundation
1:23 to make an asynchronous version
1:25 but it'll still be interesting to explore.
1:28 Okay, so you can see I've pasted some code here
1:30 and we're going to use this external package called Colorama.
1:33 So I'm going to create a requirements file
1:36 just so you can clearly see that you need to have this.
1:43 We'll come down here, pip install -r requirements.txt
1:47 into our virtual environment that we created.
1:50 All right, everything is looking good.
1:52 Now, let's talk about our program here.
1:54 So we have this main method.
1:57 And the main method is going to do sort of
1:59 this orchestration thing.
2:01 So it's going to print out that it's started
2:03 and we're using Colorama to make sure that is white.
2:06 And I'm also flushing it so that you
2:08 immediately see the output, we're flushing everything
2:10 so that you immediately see the output.
2:12 This is important because the interweaving
2:14 of behaviors from the different parts of our program
2:16 are really important to detect.
2:19 Okay, so what we're going to do is, we're going to implement this pattern
2:21 called the producer-consumer pattern.
2:24 Now, the idea is that there's
2:26 some part of our system, maybe it's even a web service
2:28 that's receiving an external request.
2:29 It's going to generate stuff to be processed
2:31 and basically put into a queue or a list
2:34 this data thing here, in this case.
2:38 And then another part, once the data's been generated
2:39 is going to take that data and process it.
2:43 So if we look down here, you can see
2:46 here's the generated data and it's super basic
2:48 it just goes and creates a square of a bunch of numbers
2:49 and throws them into this list.
2:51 And it prints out yellow, the generator is yellow.
2:56 And then the processing part says
2:57 we're going to go down here and loop over this
2:59 and pull this data out.
3:01 And in cyan, a blueish, it's going to say
3:03 we processed this value.
3:05 It also computes how long from a request to generate
3:10 or the generation of data
3:11 to the actual processing of data this takes.
3:15 Let's go over here and run this in PyCharm.
3:23 You can see the app has started
3:25 the white's kind of gray in this
3:26 little terminal thing, but it's okay.
3:28 Then we're generating items
3:29 notice all of the work is being generated
3:31 all the generation is happening, and then the processing.
3:35 That's not surprising, we call generate_data
3:38 and then we call process_data, totally reasonable.
3:39 So we're going to do 20 items, and now they're processing
3:44 one, four, nine, these are the order they came in.
3:46 And notice, the first one took almost 20 seconds
3:48 until it got to be processed.
3:51 And if you go down, it gets a little bit less drawn out.
3:54 But the generation takes some time
3:56 the processing takes some time.
3:58 So you can see the best case scenario
3:59 the one that was closest, because processing
4:02 is slightly quicker than generating
4:04 it still took 10 seconds to get to that request.
4:09 And notice, it all happened, all
4:10 the generation, all the processing.
4:14 Let's just look really quickly
4:15 at what we're calling it up at the top
4:16 that's totally reasonable, right?
4:17 We called generate_data 20 times
4:19 and we did 20, and then we called process_data
4:22 and we processed 20 at a time.
4:24 But if we look at either of these
4:26 there's a part where we're waiting.
4:29 And this simulates requests coming in
4:30 or we're waiting on an external system
4:32 without adding the complexity of something like that
4:35 which we could build, but would be way overkill
4:38 in terms of trying to demonstrate this.
4:40 This is just simulating, it takes some time
4:44 to do some work and that time probably
4:46 depends on an external system, right?
4:47 We're waiting on something, right?
4:49 Waiting on a database, waiting on a web service.
4:51 We're not really doing that
4:52 so we're just calling, time.sleep
4:54 which puts our thread to sleep.
4:57 But it means that it takes half a second
5:01 at least half a second, up to 1.5
5:03 seconds to generate an item.
5:05 Same thing, we're processing the data
5:07 we're pulling out just basically holding onto it
5:10 and talking about processing it
5:12 and then it takes us half a second to process it.
5:16 Okay, so this is this program.
5:18 This is the synchronous version of what we're doing.
5:20 And we've introduced via time these
5:23 sort of arbitrary wait periods, but
5:24 they're very much modeling real world things
5:27 like databases and web services and so on.
5:30 How can we fix this?
5:32 How can we make this better with asynchronous programming?
5:35 Well, if we could say, after we've generated a few items
5:39 while we're not generating, we're just waiting
5:41 maybe the processing could kick in and process
5:43 the few we've already generated.
5:46 But if we're not processing any
5:48 because we're waiting down here, we could generate some more.
5:51 These could work as a team, but they don't.
5:53 They don't because we wrote this
5:54 very straightforward program.
5:57 Grab the time, print out some colored stuff
6:00 but really that's not so interesting.
6:01 This, this is all there is to it.
6:03 We said, generate_data, and then process_data.
6:07 So we're going to generate it and wait, and then process it.
6:10 And that's the way synchronous works.
6:12 But we can do better if we write a different version.