Async Techniques and Examples in Python Transcripts
Chapter: Welcome to the course
Lecture: Topics covered
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's talk about the actual topics that we're going to cover chapter by chapter and how it all fits together.
0:07
We're going to start digging further into why do we care about async, when should we use it, what are its benefits.
0:13
So we're going to go way into it so you understand all the ways in which we might use asynchronous programming
0:17
in Python, and when you might want to do that. Then it's time to start writing some code and making things concrete.
0:24
We're going to focus first on the new key words introduced in Python 3.5 async and await. Now some courses leave this to the end
0:32
as the great build-up, but I think you start here. This is the new, powerful way to do threading for anything that is waiting.
0:40
Are you calling it database? Are you talking to a web service? Are you talking to the file system, things like that?
0:46
We do these kinds of things all the time in Python and it' really not productive to just block our program
0:51
while it's happening. We could be doing many other things. And the async and await key words in the asyncio foundation make this super straightforward.
1:00
It's almost exactly the same programming model as the serial version but it's way more scalable and productive.
1:07
Next we're going to focus on threads, sticking to making a single process more concurrent doing more at once.
1:13
We're going to talk about a more traditional way of writing asynchronous code in Python with threads. We'll see sometimes this is super-productive.
1:22
other times it's not as productive as you might hope especially because of things like the GIL raise its head.
1:28
And we'll see when and how to deal with that. Some things are well-addressed with threads others not so much. When we start writing multithreaded code
1:36
or asynchronous code, in general we have to think very carefully about the data structures that we use
1:42
and making sure that we don't encourage a race conditions or deadlocks. So in this chapter we're going to talk about both of those.
1:49
How do we prevent race conditions that might allow code to see invalid data or corrupt our data structures? And how do we prevent deadlocks
1:56
from completely freezing up our program by the improper use of the tools trying to prevent the first? So we're going to talk a lot about thread safety
2:03
make sure you get that just right. Now Python has two traditional types of parallelism threaded parallelism and process-based parallelism.
2:11
And the primary reason we have this is because of the GIL. We'll see that threaded-based parallelism
2:17
is great when you're waiting on things like databases or web calls, things like that. But it's basically useless for computational work.
2:25
So if you wanta do something computational we're going to have to employ process-based parallelism.
2:31
We're going to talk about Python's native, multiprocessing process-based parallelism, with tools all around that meant to take a bunch of work
2:39
and spread it across processes. You'll see that the API for working with threads and the API for working with processes are not the same.
2:47
But the execution pools are ways to unify these things so that our actual algorithms or actual code depend as little as possible on the APIs
2:56
for either threads or processes, meaning we can switch between threads or processes depending on what we're doing.
3:02
So we wanta talk about execution pools and how to unify those two APIs. Then we're going to see two really interesting libraries
3:12
that take async and await and asyncio and make it better make it easier to fall into the pit of success.
3:19
You just do the right thing, and it just happens. The way it guides you, things work better. So things like cancellation, parent/child tasks
3:27
or any mix mode of, say, some IO-boundwork and some CPU boundwork. That can be really tricky, and we'll see some libraries
3:35
that make it absolutely straightforward and obvious. One of the great places we would like to ply asyncio is on the web.
3:43
That's a place where we're waiting on databases and other web services all the time. We'll see the traditional, popular frameworks
3:50
like Django, Flask, Pyramid do not support any form of asynchrony on the web. So we'll take something that is a Flask-like API
3:59
and adapt it to use asyncio and it's going to be really, really great. We'll see massive performance improvements around our web app there.
4:06
Finally, we'll see that we can integrate C with Python and, as you know, C can do just about anything. Your operating system is written in C.
4:15
It can do whatever it wants. So we'll see that C is actually a gateway to a different aspect, different type of parallelism and performance in Python.
4:24
But we don't wanta write C. Maybe you do, but most people don't want to write C if they're already writing Python.
4:29
So we'll see that we can use something called Cython to bridge the gap between C and Python and Cython has special key words to unlock C's parallelism
4:39
in the Python interpreter. It's going to be great. So this is what we're covering during this course and I think it covers the gamut
4:46
of what Python has to offer for asynchronous programming. There's so much here; I can't wait to get started sharin' it with you.