Python Memory Management and Tips Transcripts
Chapter: Python variables and memory
Lecture: Red pill / blue pill
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
I'm going to reveal the hidden truth about Python variables to you now. So, if you've seen the movie The Matrix,
0:07
one of the best science fiction movies of all time, Morpheus spoke to Neo when he was just first realizing,
0:14
or was being told, that he was living in a simulation and Morpheus told him a little bit about that and said, "look, you have two choices here,
0:23
a choice between two things. One. You can take the blue pill.
0:27
Forget this happened and just go back to being blissfully unaware that you're actually in this
0:32
weird, dystopian world. Or you can take the red pill and see the truth. You can see inside the simulation what's actually happening".
0:40
And I feel a lot of this course is a little bit like that. When we work with Python, it generally just works, everything's smooth.
0:49
Sometimes it uses more memory than we want. Well, that's the way it is. There's not a whole lot we can do about it,
0:54
right? What I'm gonna show you in this course, is there are a lot of little techniques that combine together that will either help you build
1:01
better data structures, better algorithms and how you use those data structures,
1:06
or at least just understand why your program is using a lot of memory or it's a little bit slow or whatever. And in order to do that,
1:14
we have to look beyond the Python syntax down into the CPython runtime. So in that regard, this course is much like taking the red pill.
1:23
By the way, if you wanna watch this little segment, the link at the bottom is like a five minute video. It's great. So let's take the red pill.
1:31
Here's some Python code. Age equals 42, so age in an integer. Yes, its value is 42. You can add it. You can divide it. You can treat it like, you know,
1:41
numbers in a programming language. It couldn't be simpler, right? Well, not exactly.
1:48
So if we look at what is actually happening inside of the C Python runtime, wouldn't we work with numbers like these integer types,
1:58
They're called integer types or int in Python, but they're actually PyLong objects in the C level.
2:05
You'll see there's a whole lot of stuff going on here. Now, this is just a very small part of a single function in a very large piece
2:13
of code that you can actually click "Go here" and go to bit.ly/cPythonlong. This will take you to this line in the
2:22
CPython source code, which I don't remember exactly how long it is, but it's hundreds of lines long.
2:27
So the ability to create one of these things is not super simple. Now, for the number 42 it gets treated special.
2:33
Small numbers get treated special, as we'll see later. If this was 1042 it would be closer to what's actually happening.
2:40
The important thing, Is not which one of these functions around Python integers runs, but, just take this one as an example. So, PyLong_FromLong,
2:51
which is a C++ or C long that's converted to this PyLong, what does it return? It returns a PyObject pointer. Everything in Python is a PyObject
3:02
pointer. Strings, numbers, functions, source code, classes you create, everything. This is a common base class for everything in Python,
3:14
okay? But specifically what it's creating is a PyLong object that's then being sort of down casted to this lower version. So you can look through.
3:23
There's a bunch of stuff even within this function I removed just so it would fit on the screen. But it's allocating here. See this line that says
3:31
_PyLong_New(1). And then it does a bunch of work to it, is this pointer that's allocated? That's the dynamic memory allocation out on the heap.
3:40
It does a bunch of stuff to set its value, and then it returns it as a pointer,
3:44
which the Python runtime just converts that to something that feels like a nice little clean integer like that one line we have above.
3:52
But there's actually a ton going on. So this is the red pill world that we're going to explore what's happening behind the
3:59
scenes, the algorithms that are running, the reasons that they're happening throughout this course.