Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Python async landscape: Execution pools
0:00 We're going to touch on a theme
0:01 that we're going to actually hit
0:02 several times throughout this course
0:04 unifying these different APIs.
0:07 We've worked with threading
0:08 and we've worked with multiprocessing
0:10 but we saw that each API is different.
0:12 It would be better if they were not different.
0:14 If there was some way to just flip a switch
0:16 and have automatically the same API do one or the other
0:21 even better if it could figure out which to do.
0:23 We'll get to that later.
0:25 For now, we're going to come back to this place
0:27 and straddle this middle line here.
0:29 We're going to say, maybe we want to use threads
0:32 they are more lightweight, typically
0:34 and they have quicker, easier access
0:37 to the entire memory space of any particular program.
0:41 It is the same process and so pointers and memory
0:44 and variables, all that kind of stuff are perfectly shared.
0:47 Maybe though, we want to somehow
0:49 take advantage of multiprocessing
0:52 use separate processes, get around the GIL
0:55 things like that.
0:56 It would be nice if this decision was a decision we made
0:59 and not an entire API that we worked with.
1:02 That's what we're going to talk about in this chapter.
1:05 Here's what we've done so far.
1:07 For our multithreaded example
1:08 we created a list of threads
1:10 we called created thread objects.
1:12 We set the target pass the arguments
1:14 and said they were daemons.
1:15 And then we started those threads
1:18 and we waited on those threads.
1:19 It was fine, nothing too hard about that.
1:21 For multiprocessing we used a Pool.
1:23 We called apply_async() and then we closed the pool
1:25 and we joined on it to block
1:27 like we are in the last line there.
1:30 These are doing effectively the same thing.
1:33 Granted, one is running threads
1:34 one is running multiprocessing
1:36 but that decision shouldn't be massively different, right?
1:39 We could see the function parameters and what not
1:41 are similar but they're definitely not the same.
1:44 So what we're going to do is actually
1:46 talk about something that will unify this.
1:48 So let's go see that in the demo.