Async Techniques and Examples in Python Transcripts
Chapter: Common APIs with execution pools
Lecture: Demo: Executor app (process-edition)
0:00 So you've seen that we've taken our little app
0:02 that goes against these titles from these URLs
0:05 using threads via the ThreadPoolExecutor
0:08 to do all the work.
0:10 So we want to switch from maybe using threads
0:13 to using processes, using multiprocessing.
0:16 How much work is that going to be? Watch this.
0:18 We're going to go over here and say, you know what?
0:19 We're going to go from process, ProcessPoolExecutor.
0:26 Now because we imported it as an alias
0:29 down here, this line on line 19, doesn't have to change.
0:33 Before we added line five, that meant
0:35 create a ThreadPoolExecutor.
0:37 Now we've changed what that means
0:39 now it means create a ProcessPoolExecutor.
0:41 I'll go ahead and comment this out
0:42 even though, technically it'll work the same.
0:45 Now what happens if we run it?
0:47 Once again, let's make that bigger.
0:49 So it's running. You can see the same basic process.
0:53 We kick off all of the work.
0:55 We wait for a second to get all of the results.
0:58 Beautiful, and we do it all in parallel.
1:00 Notice now though, now the process ID is different.
1:04 The process ID is different for each one of the these
1:07 and it says process name is fork process one
1:10 fork process two, fork process three, and so on.
1:13 Even though we're calling across these processes
1:15 we're able to get a return value back
1:17 by just getting access to the result.
1:20 So run it one more time. Boom, quite nice, right?
1:24 So look at that.
1:25 To create the process pool edition
1:27 instead of the threaded pool edition
1:28 we change line four to line five.
1:31 That's it because we can use this base API
1:35 that is this executor API, and these futures
1:38 that come back regardless of what type of work
1:40 they're doing, it means we can use this same API
1:43 for multiprocessing and for threading. Pretty awesome.