Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Python async landscape: Execution pools
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
We're going to touch on a theme that we're going to actually hit several times throughout this course unifying these different APIs.
0:08
We've worked with threading and we've worked with multiprocessing but we saw that each API is different. It would be better if they were not different.
0:15
If there was some way to just flip a switch and have automatically the same API do one or the other even better if it could figure out which to do.
0:24
We'll get to that later. For now, we're going to come back to this place and straddle this middle line here.
0:30
We're going to say, maybe we want to use threads they are more lightweight, typically and they have quicker, easier access
0:38
to the entire memory space of any particular program. It is the same process and so pointers and memory
0:45
and variables, all that kind of stuff are perfectly shared. Maybe though, we want to somehow take advantage of multiprocessing
0:53
use separate processes, get around the GIL things like that. It would be nice if this decision was a decision we made
1:00
and not an entire API that we worked with. That's what we're going to talk about in this chapter. Here's what we've done so far.
1:08
For our multithreaded example we created a list of threads we called created thread objects. We set the target pass the arguments
1:15
and said they were daemons. And then we started those threads and we waited on those threads. It was fine, nothing too hard about that.
1:22
For multiprocessing we used a Pool. We called apply_async() and then we closed the pool and we joined on it to block
1:28
like we are in the last line there. These are doing effectively the same thing. Granted, one is running threads one is running multiprocessing
1:37
but that decision shouldn't be massively different, right? We could see the function parameters and what not
1:42
are similar but they're definitely not the same. So what we're going to do is actually talk about something that will unify this.
1:49
So let's go see that in the demo.