Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Concept: Execution pools
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
So we've seen the threading API and the multiprocessing API are not the same. They're basically isomorphic.
0:08
You could map from one to the other pretty easily but they're not the same. It would be nice if they were actually exactly the same
0:15
then we could switch between one or the other. And that's what we've just seen. Enter the executor. Over here we are using the ThreadPoolExecutor
0:23
which allows us to create a with statement. And inside that with statement we can execute a whole bunch of asynchronous work.
0:31
When you leave the with statement, the work is done. Basically the with statement blocks while this is happening.
0:37
Notice this is not like the pool in multiprocessing which you had to make sure you closed it and you joined on it
0:42
and you had to do those in the right order. This is a little bit simpler. So we've got this pool here and we're creating an instance of the executor
0:50
and that's where it's going to run. And then we just go to the executor and we submit work passing the function to run
0:55
and the arguments that we're going to pass to it. This comes back as a Future which we're keeping track of in a list
1:01
and then we can look at all the results when we're done with all the work. You could look at them along the way and things like that
1:06
but that's more complicated and we're just not interested in it right now. A really simple API cleans up everything
1:12
and I think it's one of the better APIs we have around threading in Python. And the big benefit which we've been hitting on here
1:20
is that we can choose our implementation. This one uses threads. If you don't like threads, change that one line from
1:27
concurrent.futures.thread import ThreadPoolExecutor concurrent.futures.process import ProcessPoolExecutor And because we used an alias
1:37
it makes it even simpler for our code to not change except for on that one import line. Beautiful.