Async Techniques and Examples in Python Transcripts
Chapter: Welcome to the course
Lecture: Topics covered
0:00 Let's talk about the actual topics
0:02 that we're going to cover chapter by chapter
0:04 and how it all fits together.
0:06 We're going to start digging further into why do we care
0:09 about async, when should we use it, what are its benefits.
0:12 So we're going to go way into it so you understand
0:14 all the ways in which we might use asynchronous programming
0:16 in Python, and when you might want to do that.
0:19 Then it's time to start writing some code
0:21 and making things concrete.
0:23 We're going to focus first on the new key words introduced
0:26 in Python 3.5 async and await.
0:29 Now some courses leave this to the end
0:31 as the great build-up, but I think you start here.
0:34 This is the new, powerful way to do threading
0:37 for anything that is waiting.
0:39 Are you calling it database?
0:41 Are you talking to a web service?
0:42 Are you talking to the file system, things like that?
0:45 We do these kinds of things all the time in Python
0:47 and it' really not productive to just block our program
0:50 while it's happening. We could be doing many other things.
0:52 And the async and await key words in the asyncio foundation
0:57 make this super straightforward.
0:59 It's almost exactly the same programming model
1:01 as the serial version
1:03 but it's way more scalable and productive.
1:06 Next we're going to focus on threads, sticking
1:08 to making a single process more concurrent
1:11 doing more at once.
1:12 We're going to talk about a more traditional way
1:15 of writing asynchronous code in Python with threads.
1:18 We'll see sometimes this is super-productive.
1:21 other times it's not as productive as you might hope
1:24 especially because of things like the GIL raise its head.
1:27 And we'll see when and how to deal with that.
1:29 Some things are well-addressed with threads
1:31 others not so much.
1:33 When we start writing multithreaded code
1:35 or asynchronous code, in general
1:37 we have to think very carefully
1:38 about the data structures that we use
1:41 and making sure that we don't encourage a race conditions
1:43 or deadlocks.
1:44 So in this chapter we're going to talk about both of those.
1:48 How do we prevent race conditions that might allow code
1:51 to see invalid data or corrupt our data structures?
1:53 And how do we prevent deadlocks
1:55 from completely freezing up our program
1:57 by the improper use of the tools trying
1:59 to prevent the first?
2:01 So we're going to talk a lot about thread safety
2:02 make sure you get that just right.
2:04 Now Python has two traditional types of parallelism
2:07 threaded parallelism and process-based parallelism.
2:10 And the primary reason we have this is because of the GIL.
2:14 We'll see that threaded-based parallelism
2:16 is great when you're waiting on things like databases
2:18 or web calls, things like that.
2:20 But it's basically useless for computational work.
2:24 So if you wanta do something computational
2:27 we're going to have to employ process-based parallelism.
2:30 We're going to talk about Python's native, multiprocessing
2:33 process-based parallelism, with tools all around
2:37 that meant to take a bunch of work
2:38 and spread it across processes.
2:40 You'll see that the API for working with threads and the API
2:43 for working with processes are not the same.
2:46 But the execution pools are ways to unify these things
2:50 so that our actual algorithms or actual code depend
2:53 as little as possible on the APIs
2:55 for either threads or processes, meaning we can switch
2:58 between threads or processes depending on what we're doing.
3:01 So we wanta talk about execution pools
3:03 and how to unify those two APIs.
3:07 Then we're going to see two really interesting libraries
3:11 that take async and await and asyncio and make it better
3:15 make it easier to fall into the pit of success.
3:18 You just do the right thing, and it just happens.
3:21 The way it guides you, things work better.
3:23 So things like cancellation, parent/child tasks
3:26 or any mix mode of, say, some IO-boundwork
3:30 and some CPU boundwork.
3:32 That can be really tricky, and we'll see some libraries
3:34 that make it absolutely straightforward and obvious.
3:37 One of the great places we would like
3:39 to ply asyncio is on the web.
3:42 That's a place where we're waiting on databases
3:44 and other web services all the time.
3:46 We'll see the traditional, popular frameworks
3:49 like Django, Flask, Pyramid do not support any form
3:53 of asynchrony on the web.
3:55 So we'll take something that is a Flask-like API
3:58 and adapt it to use asyncio
4:00 and it's going to be really, really great.
4:01 We'll see massive performance improvements
4:03 around our web app there.
4:05 Finally, we'll see that we can integrate C with Python
4:09 and, as you know, C can do just about anything.
4:12 Your operating system is written in C.
4:14 It can do whatever it wants.
4:16 So we'll see that C is actually a gateway
4:18 to a different aspect, different type
4:21 of parallelism and performance in Python.
4:23 But we don't wanta write C.
4:25 Maybe you do, but most people don't want to write C
4:27 if they're already writing Python.
4:28 So we'll see that we can use something called Cython
4:31 to bridge the gap between C and Python
4:33 and Cython has special key words to unlock C's parallelism
4:38 in the Python interpreter.
4:40 It's going to be great.
4:41 So this is what we're covering during this course
4:43 and I think it covers the gamut
4:45 of what Python has to offer for asynchronous programming.
4:49 There's so much here; I can't wait
4:51 to get started sharin' it with you.