Async Techniques and Examples in Python Transcripts
Chapter: Why async?
Lecture: An upper bound for async speed improvement
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Now you may be thinking, "Oh my gosh, Michael you have 12 CPUs, you can make your code go 12 times faster."
0:09
Sorry to tell you that is almost never true. Every now and then you'll run into algorithms that are sometimes referred to
0:16
as embarrassingly parallelizable. If you do, say, ray tracing, and every single pixel is going to have it's own track of computation.
0:25
Yes, we could probably make that go nearly 12 times faster. But most algorithms, and most code, doesn't work that way.
0:33
So if we look at maybe the execution of a particular algorithm, we have these two sections here, these two greens sections
0:40
that are potentially concurrent. Right now they're not, but imagine when we said "Oh that section and this other section.
0:46
We could do that concurrently." And let's say those represent 15% and 5% of the overall time. If we were able to take this code
0:55
and run it on an algorithm that entirely broke that up into parallelism, the green parts. Let's say the orange part cannot be broken apart.
1:03
We'll talk about why that is in just a second. If we can break up this green part and let's imagine we had as many cores
1:09
as we want, a distributed system on some cloud system. We could add millions of cores if we want. Then we could make those go to zero.
1:16
And if we could make the green parts go to zero like an extreme, non-realistic experience but think of it as a upper bound then how much would be left?
1:26
80%, the overall performance boost we could get would only be 20%. So when you're thinking about concurrency you need to think about, well how much
1:34
can be made concurrent and is that worth the added complexity? And the added challenges, as we'll see.
1:42
Maybe it is. It very well may be. But it might not be. In this case, maybe a 20% gain but really added complexity.
1:50
Maybe it's not worth it. Remember that 20% is a gain of if we could add infinite parallelism basically, to make that go to zero
1:57
which won't really happen, right? So you want to think about what is the upper bound and why might there might be orange sections?
2:04
Why might there be sections that we just can't execute in parallel? Let's think about how you got in this course.
2:11
You probably went to the website, and you found the course and you clicked a button and said, "I'd like to buy this course,"
2:14
put in you credit card, and the system went through a series of steps. It said, "Well, OK, this person wants to buy the course.
2:19
"Here's their credit card. We're going charge their card then we're going to record an entry in the database that says they're in the course
2:26
and then we're going to send an email that says, hey, thanks for buying the course here's your receipt, go check it out."
2:32
That can't really be parallelized. Maybe the last two.Maybe if you're willing to accept that email going out potentially
2:39
if the database failed, it's unlikely but, you know, possible. But you certainly cannot take charging the credit card and sending the welcome email
2:48
and make those run concurrently. There's a decent chance that for some reason a credit card got typed in wrong
2:54
it's flagged for fraud, possibly not appropriately but, right, you got to see what the credit card system and the company says.
3:03
There might not be funds for this for whatever reason. So we just have to first charge the credit card and then send the email.
3:10
There's no way to do that in parallel. Maybe we can do it in a way that's more scalable that lets other operations unrelated to this run.
3:18
That's a different consideration but in terms of making this one request this one series of operations faster we can't make those parallel.
3:27
And that's the orange sections here. Just, a lot of code has to happen in order and that's just how it is.
3:33
Every now and then, though, we have these green sections that we can parallelize, and we'll be focused on that in this course.
3:39
So keep in mind there's always an upper bound for improvement, even if you had infinite cores and infinite parallelism, you can't always
3:46
just make it that many times faster, right? There's these limitations, these orange sections that have to happen in serial.