Async Techniques and Examples in Python Transcripts
Chapter: Course conclusion and review
Lecture: Review: Execution pools
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
We saw that the API for multithreading and the API for the multiprocessing although the words sound similar and the actual behavior's somewhat similar
0:10
the API is really not very similar. They're close, they're maybe isomorphic. I could convert from one to the other pretty easily.
0:20
They seem to have most of the functionality that you might need on both but they're not exactly the same. That means in your code, you have to commit
0:27
to I'm writing against the thread API or I'm writing against the multiprocessing API but it's not easy to switch. So enter the executor.
0:36
Because it's not easy to switch when you use that API but there is a new, better way called the pool executor.
0:43
So here we're going to the concurrent.futures module which is part of standard Python. It's nothing special you go get
0:51
and we're importing the ThreadPoolExecutor and I'm giving it a name here, which is nice. Because that lets us choose our implementation.
0:59
So if we write it like this we run our code below we're going to run a bunch of work and getting the titles by downloading
1:06
from the URLs, we're going to do that on threads. But with a simple import change to the ProcessPoolExecutor we're now doing that same work
1:15
on process pools. So this is really nice. It lets us use the same API and switch between multiprocessing and multithreading.
1:24
Down here we're going to create a with block and extenuate one of these executors. We're going to submit a bunch of work
1:30
and then when you leave the with block you don't leave it until the work is all done and so when you leave, you can go and get the data back from all
1:37
the actions you've submitted. It's really nice, I definitely recommend something along these lines if you're doing threading or multiprocessing.