Async Techniques and Examples in Python Transcripts
Chapter: Why async?
Lecture: An upper bound for async speed improvement

Login or purchase this course to watch this video and the rest of the course contents.
0:00 Now you may be thinking, "Oh my gosh, Michael you have 12 CPUs, you can make your code go 12 times faster."
0:09 Sorry to tell you that is almost never true. Every now and then you'll run into algorithms that are sometimes referred to
0:16 as embarrassingly parallelizable. If you do, say, ray tracing, and every single pixel is going to have it's own track of computation.
0:25 Yes, we could probably make that go nearly 12 times faster. But most algorithms, and most code, doesn't work that way.
0:33 So if we look at maybe the execution of a particular algorithm, we have these two sections here, these two greens sections
0:40 that are potentially concurrent. Right now they're not, but imagine when we said "Oh that section and this other section.
0:46 We could do that concurrently." And let's say those represent 15% and 5% of the overall time. If we were able to take this code
0:55 and run it on an algorithm that entirely broke that up into parallelism, the green parts. Let's say the orange part cannot be broken apart.
1:03 We'll talk about why that is in just a second. If we can break up this green part and let's imagine we had as many cores
1:09 as we want, a distributed system on some cloud system. We could add millions of cores if we want. Then we could make those go to zero.
1:16 And if we could make the green parts go to zero like an extreme, non-realistic experience but think of it as a upper bound then how much would be left?
1:26 80%, the overall performance boost we could get would only be 20%. So when you're thinking about concurrency you need to think about, well how much
1:34 can be made concurrent and is that worth the added complexity? And the added challenges, as we'll see.
1:42 Maybe it is. It very well may be. But it might not be. In this case, maybe a 20% gain but really added complexity.
1:50 Maybe it's not worth it. Remember that 20% is a gain of if we could add infinite parallelism basically, to make that go to zero
1:57 which won't really happen, right? So you want to think about what is the upper bound and why might there might be orange sections?
2:04 Why might there be sections that we just can't execute in parallel? Let's think about how you got in this course.
2:11 You probably went to the website, and you found the course and you clicked a button and said, "I'd like to buy this course,"
2:14 put in you credit card, and the system went through a series of steps. It said, "Well, OK, this person wants to buy the course.
2:19 "Here's their credit card. We're going charge their card then we're going to record an entry in the database that says they're in the course
2:26 and then we're going to send an email that says, hey, thanks for buying the course here's your receipt, go check it out."
2:32 That can't really be parallelized. Maybe the last two.Maybe if you're willing to accept that email going out potentially
2:39 if the database failed, it's unlikely but, you know, possible. But you certainly cannot take charging the credit card and sending the welcome email
2:48 and make those run concurrently. There's a decent chance that for some reason a credit card got typed in wrong
2:54 it's flagged for fraud, possibly not appropriately but, right, you got to see what the credit card system and the company says.
3:03 There might not be funds for this for whatever reason. So we just have to first charge the credit card and then send the email.
3:10 There's no way to do that in parallel. Maybe we can do it in a way that's more scalable that lets other operations unrelated to this run.
3:18 That's a different consideration but in terms of making this one request this one series of operations faster we can't make those parallel.
3:27 And that's the orange sections here. Just, a lot of code has to happen in order and that's just how it is.
3:33 Every now and then, though, we have these green sections that we can parallelize, and we'll be focused on that in this course.
3:39 So keep in mind there's always an upper bound for improvement, even if you had infinite cores and infinite parallelism, you can't always
3:46 just make it that many times faster, right? There's these limitations, these orange sections that have to happen in serial.

Talk Python's Mastodon Michael Kennedy's Mastodon