Async Techniques and Examples in Python Transcripts
Chapter: Course conclusion and review
Lecture: Review: Cython
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
We say that Python threads do not really add any benefit whatsoever if they're CPU bound so if most of their operations are actually Python
0:09
instructions, not just waiting on a web service call or something on the network or file system that's because of the GIL.
0:16
The GIL means in this Python process only one interpreter or instruction will execute at a time, it doesn't matter how
0:23
many threads there are, which one's running one at a time. Okay, so that's the GIL. We saw that if it's Cython, we can rewrite our method in C
0:33
because Cython takes Python and compiles it to C. And we can also use a nogil keyword which tells CPython, everything we're doing here has nothing
0:42
to do with Python, it's pure C so it's just going to operate down here. You go let other stuff run and when this comes back we'll reacquire the GIL.
0:50
So all you have to do is say with gil and then you do your computational work. It sounds simpler than it is though because
0:57
you have to make sure that you're not interacting with any Python objects, you're going to capture all those values and then use them basically in C.
1:05
And that's even down to the square root method. So we were using math.sqrt, but that was a Python method so it accepted a py object type thing
1:14
and that was going to work in the nogil world. So we had to use the libc.math sqrt and remember we used cimport, not regular import there.
1:23
But once we did this, I saw like 1,000 times improvement in our code, it was really, really worth it for what we were doing.
1:32
So Cython wouldn't be the first thing I jump at but if its solution fits your use case it seems like a really good option.