Async Techniques and Examples in Python Transcripts
Chapter: Course conclusion and review
Lecture: Review: Execution pools
0:00 We saw that the API for multithreading
0:02 and the API for the multiprocessing
0:05 although the words sound similar
0:06 and the actual behavior's somewhat similar
0:09 the API is really not very similar.
0:12 They're close, they're maybe isomorphic.
0:16 I could convert from one to the other pretty easily.
0:19 They seem to have most of the functionality
0:20 that you might need on both
0:22 but they're not exactly the same.
0:24 That means in your code, you have to commit
0:26 to I'm writing against the thread API
0:29 or I'm writing against the multiprocessing API
0:31 but it's not easy to switch.
0:33 So enter the executor.
0:35 Because it's not easy to switch when you use that API
0:38 but there is a new, better way
0:40 called the pool executor.
0:42 So here we're going to the concurrent.futures module
0:46 which is part of standard Python.
0:48 It's nothing special you go get
0:50 and we're importing the ThreadPoolExecutor
0:53 and I'm giving it a name here, which is nice.
0:55 Because that lets us choose our implementation.
0:58 So if we write it like this
1:00 we run our code below
1:01 we're going to run a bunch of work
1:03 and getting the titles by downloading
1:05 from the URLs, we're going to do that on threads.
1:08 But with a simple import change
1:11 to the ProcessPoolExecutor
1:12 we're now doing that same work
1:14 on process pools.
1:16 So this is really nice.
1:17 It lets us use the same API and switch
1:20 between multiprocessing and multithreading.
1:23 Down here we're going to create a with block
1:25 and extenuate one of these executors.
1:27 We're going to submit a bunch of work
1:29 and then when you leave the with block
1:31 you don't leave it until the work is all done
1:33 and so when you leave, you can go
1:34 and get the data back from all
1:36 the actions you've submitted.
1:38 It's really nice, I definitely recommend
1:39 something along these lines if you're doing
1:42 threading or multiprocessing.