Async Techniques and Examples in Python Transcripts
Chapter: Why async?
Lecture: Why threads don't perform in Python
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
I mentioned earlier that threads in Python don't really perform well. They're fine if you're waiting but if you're doing computational stuff
0:09
they really don't help. Remember that example I showed you how we had our Python program doing a tight while loop that was just pounding the CPU
0:17
and it's only getting 8.3% CPU usage. Well if we had added 12 threads to do that how much of a benefit would it have gotten?
0:25
Zero, it would still only be using 8% of the CPU, even though I have 12 cores. And the reason is this thing called the GIL
0:34
or the Global Interpreter Lock. Now the GIL is often referred to as one of the reasons that Python isn't super fast or scalable.
0:42
I think that's being a little bit harsh. You'll see there's a lot of places where Python can run concurrently and can do interesting parallel work.
0:51
And it has a lot of things that have been added like AsyncIO and the await, the async and await keywords which are super, super powerful.
0:59
The GIL is not the great terrible thing that people have made it out to be but it does mean that there's certain situations
1:06
where Python cannot run concurrently. And that's a bummer. So this is the reason that the Python threads
1:13
don't perform well. The Global Interpreter Lock means only one thread or only one step of execution in Python can ever run at the same time
1:22
regardless of whether that's in the same thread or multiple threads, right? The Python interpreter only processes instructions
1:30
one at a time, no matter where they come from. You might think of this as a threading thing and in some sense it's a thread safety thing
1:37
but the GIL is actually a memory management feature. The reason it's there, is it allows the reference counting
1:43
which is how Python primarily handles its memory management, to be simpler and faster. With the Global Interpreter Lock
1:52
that means we don't have to take many fine-grained locks around allocation and memory management. And it means single-threaded Python is faster
1:59
than it would otherwise be although it obviously hurts parallel computation. But the thinking is most Python code is serial
2:08
in its execution, so let's make that the best and the GIL is what it is. It's going to make our parallelism less good.
2:16
As you'll see throughout this course Python has a lot of options to get around the GIL to do true concurrent processing in Python
2:24
and we're going to talk about those but the GIL cannot be avoided and your should really understand that it exists, what it is, and primarily
2:31
that it's a thread safety feature for memory management. You can learn about it more if you want at realPython.com/Python-gil, G-I-L
2:41
and there's a great article there it goes into all the depth and the history and so on, so go check that out if you're interested in digging into it.
2:48
Keep in mind, this GIL is omnipresent and we always have to think about how it affects our parallelism.