Async Techniques and Examples in Python Transcripts
Chapter: Threads
Lecture: Demo: Something productive with threads
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's return to an old friend the producer consumer that we learned about in the asyncio chapter. So here we have the single threaded
0:11
non-concurrent producer consumer thing that we've been playing with in the previous chapter. So we're going to generate a bunch of data
0:17
and we're going to generate some more data then we're going to process it. Now if we run it, remember it generates some data the first time through
0:24
and then it does it again, and so on. However, we saw that using asyncio let us really speed this up.
0:31
Let's assume for some reason that we don't have asyncio available to us. Now what we're actually doing when we do
0:37
and we've already implemented it and you've seen that but assuming that we don't, how would we model this with threads?
0:43
A lot of the reasons I gave you before that might be the case here. Maybe we're interacting with some system here
0:49
that we're waiting on, but it doesn't actually have an asyncio option for us. So we're going to do what we just talked about
0:55
and we're going to have the threads that we're going to work with. I'm going to go in here and I'm going to create some threads
1:00
need to import threading, set the target to generate_data, and set the args to be what they're going to be, so 20 and data
1:14
and I guess if you want to use data you should probably define it above. That all looks good, and are doing that again
1:20
and then the other one, we're using process_data and we're using 40. So that's going to get rid of those. Now again, this doesn't start them.
1:31
So if we ran this now, it would be super fast zero milliseconds, zero seconds at least. However, it didn't do any work.
1:40
So what we want to do is use our little trick t.start() for t in threads, and then we want to print started, somethin' like that
1:52
and then we'll do join. Alright, so this, that should really be all there was to it, and let's try. Now this assumes of course that process_data
2:05
will go through and it has this sort of continue step to know how much it's supposed to process. There's a little bit of work to make sure this works
2:13
but it was already in place, and so we should be able to just run this now. There they go, see them producing
2:19
and consuming in parallel. Perfect, right? Looks just like the asyncio version that we were working with, except for
2:27
apparently we can't pop from an empty list let's see what we did there. Ah yes, alright, let's fix this. Alright, so that should work
2:41
sorry about that, try again. See, threading will pull out interesting errors you did not think were there. Here you can see it's done, the latency's
2:51
a little bit higher than we were looking at previously. The overall execution time is about the same
2:57
as our asyncio version. Pretty straightforward right? We just use our technique, create a whole bunch of threads start them, maybe do other work
3:06
maybe just chill, and then wait for them to finish by joining on all of them.