Async Techniques and Examples in Python Transcripts
Chapter: asyncio-based web frameworks
Lecture: A note about rate limiting with external services
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
What we're talking about with this wrk benchmarking tool is otherwise what's known as load testing. Let's hit the server with as much traffic
0:10
and as many concurrent users as we can. Just keep ramping that up until it fails. Well, it turns out the way we've implemented this is
0:18
we're not talking to a local database and then just sending responses. We're basically orchestrating external services. So by load testing our code
0:28
we're going to end up load testing those services. Alright, the harder we hit our service the harder we're going to concurrently hit those services.
0:37
Now either because those services can't take it or because those service just have built in rate limiting to not let you do that much processing
0:47
that many concurrent requests, they're going to fail. And that is going to make it seem like our app is failing but it's only because
0:53
we're hammering the external services they either can't take it or don't like it and then they fail on us. So that's a problem.
1:01
How do we do load testing with these external services without getting into trouble by hitting them too hard?
1:07
Well, you may have noticed this measured latency in seconds and if used cache data. That by the way is set right here.
1:16
I'm not going to turn it on it doesn't matter right now but you can turn it on when we do the load testing if you want.
1:22
You're going to need to turn it on or you're not going to get the right numbers more or less. So how did I come up with these numbers
1:27
and what is this doing? So I ran some simple repetitive tests against this over and over and over and figured out how long does that operation
1:36
right there take? One time it took this long and one time it took that long and you can see all of these numbers
1:42
that's how long it took for this to be done. And I said okay well that's pretty representative at least from the West coast of the US.
1:49
So lets just record those numbers and for load testing purposes we'll be able to go and actually just simulate those.
1:57
So if we say use cache data we're going to simulate calling that thing. We're going to randomly pick one of the real measured latencies
2:05
and then we're going to return some fake data just something I grabbed somewhere near Portland. I'm not sure exactly where that is.
2:11
So if you're doing any sort of load testing you have to go into the config file you're using and set use cache data true
2:19
or it's just going to freak out and you're going to end up load testing datasciencetoolkit.org which is not really what we want to do
2:26
please don't do that to them. And it turns out it fails anyways so it wrecks your test, not great. If they're not explicitly rate limiting you
2:33
maybe what they should be doing is also using asyncio and things like that. But anyway, you'll see this in each one of the services.
2:41
So the lat long service has it the sun service has just some hard coded sun data. Exact same thing, here's the measured latencies.
2:50
If you turn on this it just uses the asyncio friendly sleep and then to simulate the time that it takes to return the fake data.
2:58
I want to make sure you're aware that because we're depending on these external services for load testing, you don't want to load test them
3:04
you want to simulate what it would be like in a real system. Now granted you may not want to actually depend on them
3:10
if they can't take your load testing in a real one but this is a toy course demo, right? No harm here.