Python Memory Management and Tips Transcripts
Chapter: Python variables and memory
Lecture: Red pill / blue pill

Learn more about this course
Login or purchase this course to watch this video and the rest of the course contents.
0:00 I'm going to reveal the hidden truth about python variables to you now.
0:04 So, if you've seen the movie The Matrix,
0:06 one of the best science fiction movies of all time,
0:09 Morpheus spoke to Neo when he was just first realizing,
0:13 or was being told, that he was living in a simulation and Morpheus told him
0:19 a little bit about that and said,
0:20 "look, you have two choices here,
0:22 a choice between two things. One.
0:24 You can take the blue pill.
0:26 Forget this happened and just go back to being blissfully unaware that you're actually in this
0:31 weird, dystopian world. Or you can take the red pill and see the truth.
0:36 You can see inside the simulation what's actually happening".
0:39 And I feel a lot of this course is a little bit like that.
0:43 When we work with Python, it generally just works, everything's smooth.
0:48 Sometimes it uses more memory than we want.
0:51 Well, that's the way it is.
0:52 There's not a whole lot we can do about it,
0:53 right? What I'm gonna show you in this course,
0:55 is there are a lot of little techniques that combine together that will either help you build
1:00 better data structures, better algorithms and how you use those data structures,
1:05 or at least just understand why your program is using a lot of memory or it's
1:10 a little bit slow or whatever.
1:12 And in order to do that,
1:13 we have to look beyond the python syntax down into the CPython runtime.
1:18 So in that regard, this course is much like taking the red pill.
1:22 By the way, if you wanna watch this little segment,
1:24 the link at the bottom is like a five minute video.
1:26 It's great. So let's take the red pill.
1:30 Here's some Python code. Age equals 42, so age in an integer.
1:34 Yes, its value is 42.
1:37 You can add it. You can divide it.
1:39 You can treat it like, you know,
1:40 numbers in a programming language. It couldn't be simpler,
1:43 right? Well, not exactly.
1:47 So if we look at what is actually happening inside of the C Python runtime,
1:53 wouldn't we work with numbers like these integer types,
1:57 They're called integer types or int in Python,
2:00 but they're actually PyLong objects in the C level.
2:04 You'll see there's a whole lot of stuff going on here. Now,
2:07 this is just a very small part of a single function in a very large piece
2:12 of code that you can actually click
2:14 "Go here" and go to
2:16 bit.ly/cpythonlong. This will take you to this line in the
2:21 CPython source code, which I don't remember exactly how long it is,
2:24 but it's hundreds of lines long.
2:26 So the ability to create one of these things is not super simple.
2:29 Now, for the number 42 it gets treated special.
2:32 Small numbers get treated special, as we'll see later.
2:35 If this was 1042 it would be closer to what's actually happening.
2:39 The important thing, Is not which one of these functions around Python integers runs,
2:46 but, just take this one as an example.
2:48 So, PyLong_FromLong,
2:50 which is a C++ or C long that's converted to this PyLong, what does
2:55 it return? It returns a PyObject pointer.
2:58 Everything in Python is a PyObject
3:01 pointer. Strings, numbers, functions,
3:06 source code, classes you create, everything.
3:09 This is a common base class for everything in Python,
3:13 okay? But specifically what it's creating is a PyLong object that's then being sort
3:18 of down casted to this lower version. So you can look through.
3:22 There's a bunch of stuff even within this function
3:24 I removed just so it would fit on the screen.
3:26 But it's allocating here. See this line that says
3:30 _PyLong_New(1). And then it does a bunch of work to it,
3:33 is this pointer that's allocated?
3:36 That's the dynamic memory allocation out on the heap.
3:39 It does a bunch of stuff to set its value,
3:41 and then it returns it as a pointer,
3:43 which the Python runtime just converts that to something that feels like a nice little clean
3:49 integer like that one line we have above.
3:51 But there's actually a ton going on.
3:53 So this is the red pill world that we're going to explore what's happening behind the
3:58 scenes, the algorithms that are running, the reasons that they're happening
4:02 throughout this course.