Async Techniques and Examples in Python Transcripts
Chapter: Parallelism in C with Cython
Lecture: Demo: Fast threading with Cython (int overflow issues)
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Now, looking at this timing here I think the times are so incredibly small that the stuff we're doing around the edges is actually starting to show up.
0:09
So, I tweaked things just a little bit. First of all, I'm still concerned these numbers might overflow, so I switched
0:15
them all to float, okay, float, float, float and they're much less likely to do that there. Then over here, I said we're taking all this
0:23
time to create the threads, and then then, you know, we had this time starting like way up here, so, I decided like let's create all the threads
0:34
and then start them, and the second they're started, start the time, okay? Cause, now we're going to wait
0:39
and then we're going to print out the time here. Because, I didn't see as much of a change between the serial version and the main one.
0:46
So now, now, if we run exactly the same amount and we don't do this sort of, especially this stuff like getting the processor count, and so on.
0:54
Now, if we run it, now we have a really, really sweet performance increase here. Let's actually add one more zero.
1:02
Make that 3,000,000. One final time on this one as well. Get our new baseline number. So, 80, I'll put 80 here.
1:14
Now, we can run it, and see what we get. Whoops, If you're on the same one you get the same number. Let's run the Cython line one more time.
1:20
There we go, look at that, 2,000 times faster. 2,000. 2,200. That is so awesome. Let's just make sure that this parallelism
1:30
is having some effect, so what if we go over here and we say only use six cores instead of twelve. That may make a difference.
1:39
It's a little slow right? Actually, almost half but if we only use two cores? Three hundred, three hundred.
1:46
So, you can see the fact that we're doing threading It's definitely getting faster and faster by using threading, I mean we still have zeros
1:52
all across the board but somewhere in there are milliseconds, or microseconds, or something like that that we're we're tracking here.
1:58
So, just wanted to, You know, we're going so fast that like, the getting the processor count and actually allocating the threads
2:06
that really starts to impact our performance, right? And even if you say well, we're going to consider the starting of the threads, right, it's still
2:14
four hundred times faster than the CPython version. Really, really a big deal. And I don't have my divide up there, do I still, no.
2:21
Yeah, 416 times faster. 500 that time almost. How hard was it? Well, we took the one function right here that was a little bit slow, and we're doing
2:31
most of the work, and we converted that to Cython but most of our program, all the stuff that you've learned about, threading
2:38
the regular methods, while, you know, for loops, everything. It's all the same, it's unchanged. We can use exactly the same program.
2:47
We just took the one little hotspot and replaced it with a Cython implementation. It's really nice, isn't it?