Async Techniques and Examples in Python Transcripts
Chapter: Why async?
Lecture: Concept: Visualizing a synchronous request
0:00 Let's visualize this concept of scalability
0:03 around a synchronous series of executions.
0:06 So many web servers have built in threaded modes
0:10 for Python and to some degree that helps
0:13 but it just doesn't help as much as it potentially could.
0:18 So we're going to employ other techniques
0:19 to make it even better.
0:20 So let's see what happens if we have several requests
0:22 coming into a system that executes them synchronously
0:25 one after another.
0:27 So a request comes in, request 1, request 2
0:29 and request 3 pretty much right away.
0:31 In the green boxes request 1, 2 and 3
0:34 those tell you how long those requests take.
0:38 So request 1 and request 2, they look like maybe
0:41 they're hitting the same page.
0:42 They're doing about the same thing.
0:43 3's actually really short.
0:45 Could theoretically if it were to move to the front
0:47 be needing a return much quicker.
0:49 So let's look at how this appears from the outside.
0:52 So a request 1 comes in and it takes about that long
0:57 for the response.
0:58 But during that time request 2 came in and now
1:01 before it could even get processed we have to wait
1:03 for most of 1.
1:05 So that's actually worse because we're not scaling.
1:09 Okay, we can't do as much concurrently.
1:11 And finally if we look at 3, 3 is really bad.
1:14 So 3 came in almost the same time as 1 started
1:18 but because the two longer requests were
1:20 queued up in front of it, it took about five times
1:24 four times as long as the request should of taken
1:27 for it to respond because this system was all clogged up.
1:30 Now let's zoom into one of these requests
1:32 into request 1 and if we look deep inside of it
1:36 we can break it down into two different operations.
1:39 We have a little bit of framework code
1:41 I'm just going to call that framework because
1:42 you can't control that, you can't do much with that, right.
1:44 This is the system actually receiving the socket data
1:48 creating the request response objects, things like that.
1:50 And then right at the beginning
1:52 maybe the first thing we want to do is say
1:54 I would like to find out what user's logged in
1:57 making this request.
1:58 So when I go to the database, I want to get
2:00 we'll maybe get a user ID or some kind of cookie
2:03 from their request and I'm going to use that cookie
2:05 to retrieve the actual object that represents this user.
2:08 Then maybe that user wants to list, say their courses.
2:12 So then we're going to do a little bit of code
2:14 to figure out what we're going to do there.
2:15 Okay, if the user's not None and we're going to make
2:18 this other database call
2:20 and that database call might take even longer
2:22 in terms of more data.
2:23 And then that response comes back.
2:25 We do a little bit of work to prepare it
2:26 we pass off to the framework template
2:29 and then we just hand it off to the web framework.
2:32 So this request took a long time but if you look
2:34 why it took a long time, it's mostly not because
2:38 it's busy doing something.
2:39 It's because it's busy waiting.
2:41 And when you think about web systems, websites
2:44 web servers and so on, they're often
2:47 waiting on something else.
2:49 Waiting on a web API they're calling
2:51 waiting on a database, waiting on the file system.
2:54 There's just a lot of waiting. Dr. Seuss would be proud.
2:57 Because this code is synchronous, we can't do anything else.
3:00 I mean theoretically, if we had a way to say
3:03 well we're just waiting on the database
3:04 go do something else, that would be great. But we don't.
3:07 We're writing synchronous codes, so we call this function
3:10 call the database query function
3:11 and we just block 'til it responds.
3:14 Even in the threaded mode, that is a tied up thread
3:18 over in the web worker process so this is not great.
3:22 This is why our scalability hurts.
3:24 If we could find a way to process request 2
3:27 and request 3 while we're waiting on the database
3:29 in these red zones, we could really ramp up our scalability.