Python Memory Management and Tips Transcripts
Chapter: Python variables and memory
Lecture: Red pill / blue pill
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
I'm going to reveal the hidden truth about python variables to you now.
0:04
So, if you've seen the movie The Matrix,
0:06
one of the best science fiction movies of all time,
0:09
Morpheus spoke to Neo when he was just first realizing,
0:13
or was being told, that he was living in a simulation and Morpheus told him
0:19
a little bit about that and said,
0:20
"look, you have two choices here,
0:22
a choice between two things. One.
0:24
You can take the blue pill.
0:26
Forget this happened and just go back to being blissfully unaware that you're actually in this
0:31
weird, dystopian world. Or you can take the red pill and see the truth.
0:36
You can see inside the simulation what's actually happening".
0:39
And I feel a lot of this course is a little bit like that.
0:43
When we work with Python, it generally just works, everything's smooth.
0:48
Sometimes it uses more memory than we want.
0:51
Well, that's the way it is.
0:52
There's not a whole lot we can do about it,
0:53
right? What I'm gonna show you in this course,
0:55
is there are a lot of little techniques that combine together that will either help you build
1:00
better data structures, better algorithms and how you use those data structures,
1:05
or at least just understand why your program is using a lot of memory or it's
1:10
a little bit slow or whatever.
1:12
And in order to do that,
1:13
we have to look beyond the python syntax down into the CPython runtime.
1:18
So in that regard, this course is much like taking the red pill.
1:22
By the way, if you wanna watch this little segment,
1:24
the link at the bottom is like a five minute video.
1:26
It's great. So let's take the red pill.
1:30
Here's some Python code. Age equals 42, so age in an integer.
1:34
Yes, its value is 42.
1:37
You can add it. You can divide it.
1:39
You can treat it like, you know,
1:40
numbers in a programming language. It couldn't be simpler,
1:43
right? Well, not exactly.
1:47
So if we look at what is actually happening inside of the C Python runtime,
1:53
wouldn't we work with numbers like these integer types,
1:57
They're called integer types or int in Python,
2:00
but they're actually PyLong objects in the C level.
2:04
You'll see there's a whole lot of stuff going on here. Now,
2:07
this is just a very small part of a single function in a very large piece
2:12
of code that you can actually click
2:14
"Go here" and go to
2:16
bit.ly/cpythonlong. This will take you to this line in the
2:21
CPython source code, which I don't remember exactly how long it is,
2:24
but it's hundreds of lines long.
2:26
So the ability to create one of these things is not super simple.
2:29
Now, for the number 42 it gets treated special.
2:32
Small numbers get treated special, as we'll see later.
2:35
If this was 1042 it would be closer to what's actually happening.
2:39
The important thing, Is not which one of these functions around Python integers runs,
2:46
but, just take this one as an example.
2:48
So, PyLong_FromLong,
2:50
which is a C++ or C long that's converted to this PyLong, what does
2:55
it return? It returns a PyObject pointer.
2:58
Everything in Python is a PyObject
3:01
pointer. Strings, numbers, functions,
3:06
source code, classes you create, everything.
3:09
This is a common base class for everything in Python,
3:13
okay? But specifically what it's creating is a PyLong object that's then being sort
3:18
of down casted to this lower version. So you can look through.
3:22
There's a bunch of stuff even within this function
3:24
I removed just so it would fit on the screen.
3:26
But it's allocating here. See this line that says
3:30
_PyLong_New(1). And then it does a bunch of work to it,
3:33
is this pointer that's allocated?
3:36
That's the dynamic memory allocation out on the heap.
3:39
It does a bunch of stuff to set its value,
3:41
and then it returns it as a pointer,
3:43
which the Python runtime just converts that to something that feels like a nice little clean
3:49
integer like that one line we have above.
3:51
But there's actually a ton going on.
3:53
So this is the red pill world that we're going to explore what's happening behind the
3:58
scenes, the algorithms that are running, the reasons that they're happening
4:02
throughout this course.