Async Techniques and Examples in Python Transcripts
Chapter: Threads
Lecture: Demo: Something productive with threads

Login or purchase this course to watch this video and the rest of the course contents.
0:00 Let's return to an old friend the producer consumer that we learned about in the asyncio chapter. So here we have the single threaded
0:11 non-concurrent producer consumer thing that we've been playing with in the previous chapter. So we're going to generate a bunch of data
0:17 and we're going to generate some more data then we're going to process it. Now if we run it, remember it generates some data the first time through
0:24 and then it does it again, and so on. However, we saw that using asyncio let us really speed this up.
0:31 Let's assume for some reason that we don't have asyncio available to us. Now what we're actually doing when we do
0:37 and we've already implemented it and you've seen that but assuming that we don't, how would we model this with threads?
0:43 A lot of the reasons I gave you before that might be the case here. Maybe we're interacting with some system here
0:49 that we're waiting on, but it doesn't actually have an asyncio option for us. So we're going to do what we just talked about
0:55 and we're going to have the threads that we're going to work with. I'm going to go in here and I'm going to create some threads
1:00 need to import threading, set the target to generate_data, and set the args to be what they're going to be, so 20 and data
1:14 and I guess if you want to use data you should probably define it above. That all looks good, and are doing that again
1:20 and then the other one, we're using process_data and we're using 40. So that's going to get rid of those. Now again, this doesn't start them.
1:31 So if we ran this now, it would be super fast zero milliseconds, zero seconds at least. However, it didn't do any work.
1:40 So what we want to do is use our little trick t.start() for t in threads, and then we want to print started, somethin' like that
1:52 and then we'll do join. Alright, so this, that should really be all there was to it, and let's try. Now this assumes of course that process_data
2:05 will go through and it has this sort of continue step to know how much it's supposed to process. There's a little bit of work to make sure this works
2:13 but it was already in place, and so we should be able to just run this now. There they go, see them producing
2:19 and consuming in parallel. Perfect, right? Looks just like the asyncio version that we were working with, except for
2:27 apparently we can't pop from an empty list let's see what we did there. Ah yes, alright, let's fix this. Alright, so that should work
2:41 sorry about that, try again. See, threading will pull out interesting errors you did not think were there. Here you can see it's done, the latency's
2:51 a little bit higher than we were looking at previously. The overall execution time is about the same
2:57 as our asyncio version. Pretty straightforward right? We just use our technique, create a whole bunch of threads start them, maybe do other work
3:06 maybe just chill, and then wait for them to finish by joining on all of them.

Talk Python's Mastodon Michael Kennedy's Mastodon