Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Concept: Execution pools
0:00 So we've seen the threading API
0:01 and the multiprocessing API are not the same.
0:04 They're basically isomorphic.
0:07 You could map from one to the other pretty easily
0:09 but they're not the same.
0:11 It would be nice if they were actually exactly the same
0:14 then we could switch between one or the other.
0:16 And that's what we've just seen. Enter the executor.
0:19 Over here we are using the ThreadPoolExecutor
0:22 which allows us to create a with statement.
0:25 And inside that with statement
0:26 we can execute a whole bunch of asynchronous work.
0:30 When you leave the with statement, the work is done.
0:33 Basically the with statement
0:34 blocks while this is happening.
0:36 Notice this is not like the pool in multiprocessing
0:38 which you had to make sure you closed it
0:40 and you joined on it
0:41 and you had to do those in the right order.
0:42 This is a little bit simpler.
0:44 So we've got this pool here
0:46 and we're creating an instance of the executor
0:49 and that's where it's going to run.
0:50 And then we just go to the executor and we submit work
0:53 passing the function to run
0:54 and the arguments that we're going to pass to it.
0:57 This comes back as a Future
0:58 which we're keeping track of in a list
1:00 and then we can look at all the results
1:02 when we're done with all the work.
1:04 You could look at them along the way and things like that
1:05 but that's more complicated
1:06 and we're just not interested in it right now.
1:09 A really simple API cleans up everything
1:11 and I think it's one of the better APIs
1:14 we have around threading in Python.
1:16 And the big benefit which we've been hitting on here
1:19 is that we can choose our implementation.
1:21 This one uses threads.
1:23 If you don't like threads, change that one line from
1:26 concurrent.futures.thread import ThreadPoolExecutor
1:29 concurrent.futures.process import ProcessPoolExecutor
1:34 And because we used an alias
1:36 it makes it even simpler for our code to not change
1:39 except for on that one import line. Beautiful.