Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Concept: Execution pools

Login or purchase this course to watch this video and the rest of the course contents.
0:00 So we've seen the threading API and the multiprocessing API are not the same. They're basically isomorphic.
0:08 You could map from one to the other pretty easily but they're not the same. It would be nice if they were actually exactly the same
0:15 then we could switch between one or the other. And that's what we've just seen. Enter the executor. Over here we are using the ThreadPoolExecutor
0:23 which allows us to create a with statement. And inside that with statement we can execute a whole bunch of asynchronous work.
0:31 When you leave the with statement, the work is done. Basically the with statement blocks while this is happening.
0:37 Notice this is not like the pool in multiprocessing which you had to make sure you closed it and you joined on it
0:42 and you had to do those in the right order. This is a little bit simpler. So we've got this pool here and we're creating an instance of the executor
0:50 and that's where it's going to run. And then we just go to the executor and we submit work passing the function to run
0:55 and the arguments that we're going to pass to it. This comes back as a Future which we're keeping track of in a list
1:01 and then we can look at all the results when we're done with all the work. You could look at them along the way and things like that
1:06 but that's more complicated and we're just not interested in it right now. A really simple API cleans up everything
1:12 and I think it's one of the better APIs we have around threading in Python. And the big benefit which we've been hitting on here
1:20 is that we can choose our implementation. This one uses threads. If you don't like threads, change that one line from
1:27 concurrent.futures.thread import ThreadPoolExecutor concurrent.futures.process import ProcessPoolExecutor And because we used an alias
1:37 it makes it even simpler for our code to not change except for on that one import line. Beautiful.

Talk Python's Mastodon Michael Kennedy's Mastodon