Async Techniques and Examples in Python Transcripts
Chapter: Welcome to the course
Lecture: Topics covered
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's talk about the actual topics
0:02
that we're going to cover chapter by chapter
0:04
and how it all fits together.
0:06
We're going to start digging further into why do we care
0:09
about async, when should we use it, what are its benefits.
0:12
So we're going to go way into it so you understand
0:14
all the ways in which we might use asynchronous programming
0:16
in Python, and when you might want to do that.
0:19
Then it's time to start writing some code
0:21
and making things concrete.
0:23
We're going to focus first on the new key words introduced
0:26
in Python 3.5 async and await.
0:29
Now some courses leave this to the end
0:31
as the great build-up, but I think you start here.
0:34
This is the new, powerful way to do threading
0:37
for anything that is waiting.
0:39
Are you calling it database?
0:41
Are you talking to a web service?
0:42
Are you talking to the file system, things like that?
0:45
We do these kinds of things all the time in Python
0:47
and it' really not productive to just block our program
0:50
while it's happening. We could be doing many other things.
0:52
And the async and await key words in the asyncio foundation
0:57
make this super straightforward.
0:59
It's almost exactly the same programming model
1:01
as the serial version
1:03
but it's way more scalable and productive.
1:06
Next we're going to focus on threads, sticking
1:08
to making a single process more concurrent
1:11
doing more at once.
1:12
We're going to talk about a more traditional way
1:15
of writing asynchronous code in Python with threads.
1:18
We'll see sometimes this is super-productive.
1:21
other times it's not as productive as you might hope
1:24
especially because of things like the GIL raise its head.
1:27
And we'll see when and how to deal with that.
1:29
Some things are well-addressed with threads
1:31
others not so much.
1:33
When we start writing multithreaded code
1:35
or asynchronous code, in general
1:37
we have to think very carefully
1:38
about the data structures that we use
1:41
and making sure that we don't encourage a race conditions
1:43
or deadlocks.
1:44
So in this chapter we're going to talk about both of those.
1:48
How do we prevent race conditions that might allow code
1:51
to see invalid data or corrupt our data structures?
1:53
And how do we prevent deadlocks
1:55
from completely freezing up our program
1:57
by the improper use of the tools trying
1:59
to prevent the first?
2:01
So we're going to talk a lot about thread safety
2:02
make sure you get that just right.
2:04
Now Python has two traditional types of parallelism
2:07
threaded parallelism and process-based parallelism.
2:10
And the primary reason we have this is because of the GIL.
2:14
We'll see that threaded-based parallelism
2:16
is great when you're waiting on things like databases
2:18
or web calls, things like that.
2:20
But it's basically useless for computational work.
2:24
So if you wanta do something computational
2:27
we're going to have to employ process-based parallelism.
2:30
We're going to talk about Python's native, multiprocessing
2:33
process-based parallelism, with tools all around
2:37
that meant to take a bunch of work
2:38
and spread it across processes.
2:40
You'll see that the API for working with threads and the API
2:43
for working with processes are not the same.
2:46
But the execution pools are ways to unify these things
2:50
so that our actual algorithms or actual code depend
2:53
as little as possible on the APIs
2:55
for either threads or processes, meaning we can switch
2:58
between threads or processes depending on what we're doing.
3:01
So we wanta talk about execution pools
3:03
and how to unify those two APIs.
3:07
Then we're going to see two really interesting libraries
3:11
that take async and await and asyncio and make it better
3:15
make it easier to fall into the pit of success.
3:18
You just do the right thing, and it just happens.
3:21
The way it guides you, things work better.
3:23
So things like cancellation, parent/child tasks
3:26
or any mix mode of, say, some IO-boundwork
3:30
and some CPU boundwork.
3:32
That can be really tricky, and we'll see some libraries
3:34
that make it absolutely straightforward and obvious.
3:37
One of the great places we would like
3:39
to ply asyncio is on the web.
3:42
That's a place where we're waiting on databases
3:44
and other web services all the time.
3:46
We'll see the traditional, popular frameworks
3:49
like Django, Flask, Pyramid do not support any form
3:53
of asynchrony on the web.
3:55
So we'll take something that is a Flask-like API
3:58
and adapt it to use asyncio
4:00
and it's going to be really, really great.
4:01
We'll see massive performance improvements
4:03
around our web app there.
4:05
Finally, we'll see that we can integrate C with Python
4:09
and, as you know, C can do just about anything.
4:12
Your operating system is written in C.
4:14
It can do whatever it wants.
4:16
So we'll see that C is actually a gateway
4:18
to a different aspect, different type
4:21
of parallelism and performance in Python.
4:23
But we don't wanta write C.
4:25
Maybe you do, but most people don't want to write C
4:27
if they're already writing Python.
4:28
So we'll see that we can use something called Cython
4:31
to bridge the gap between C and Python
4:33
and Cython has special key words to unlock C's parallelism
4:38
in the Python interpreter.
4:40
It's going to be great.
4:41
So this is what we're covering during this course
4:43
and I think it covers the gamut
4:45
of what Python has to offer for asynchronous programming.
4:49
There's so much here; I can't wait
4:51
to get started sharin' it with you.
