Python Memory Management and Tips Transcripts
Chapter: Recovering memory in Python
Lecture: GC in action
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's look at this cycle thing in the context of our little reference counting app that
0:04
we've done, and I'm just gonna print out of the top like this running reference
0:13
counting demo, cause we're gonna have another kind and these are gonna be super similar
0:16
so let's go over here and just make a
0:19
copy, say, this is going to be
0:21
"gc", and this is gonna be the "gc demo",
0:25
and this is going to explicitly enable it. We don't have to do that,
0:28
but I explicitly disabled before so let's make sure of whatever else happened,
0:33
It's in there. So we're gonna do kind of what I talked about in the
0:37
previous videos, we're gonna have 2 things, person
0:40
1, person 2, and We're gonna say
0:43
"v1.friends.append(v2)", and vice versa,
0:47
right? These are 2 people who are friends.
0:52
Let's go and tell it that this is alright.
0:55
We're going to tell it "this is a list of friends".
0:59
There we go. Make sure it's a list.
1:02
Now, we want to have the id.
1:04
So let's have this "id1 and id2, v1 and v2" just
1:10
for the same reason we needed to keep track of those.
1:14
So, id1, now let's change this to "counts are", and id2.
1:22
We don't need our step 2, well, we're gonna change
1:27
what step 2 means. Let's go and actually set "v1 = none", and
1:31
"v2 = none", and then we'll put our reference count out again, like that,
1:40
setting it to none twice is not gonna do anything.
1:43
Let's set it to be the end of the method and see when things get cleaned
1:46
up. Maybe one of these will go away.
1:49
Maybe both. We don't know.
1:51
Let's find out. So we're gonna run this one.
1:56
Well, look at that. So we've created a new object,
1:59
doomed1 and doomed2, those are Michael and Sara,
2:03
for example, and we had up here, I'll make this more legible for you,
2:08
after this, we had one pointer to each one reference count,
2:12
now they're friends, so they pointed at each other,
2:14
that incriminated it again, and you can see now the reference counts are 2 and
2:18
2, and we threw away the original variables and went back to 1 and 1
2:22
but they're never going to get better than that because of the cycle.
2:25
So we went all the way to the end of the method and they never got
2:29
cleaned up. What happened? Well,
2:31
"gc" is non-deterministic. It's based on how the program behaved.
2:36
We're gonna talk about that in a minute.
2:37
What are the gates that it has to go through and where the rules it uses
2:41
to decide when to run, what objects to look at and so on?
2:45
But the short version is just cause these are free, the garbage collector is not constantly running
2:50
around. It can only run so often,
2:52
otherwise things would be super inefficient.
2:55
So let's go and make some stuff happen.
2:59
Both to step 3. Down here
3:02
I'm gonna just, kind of like we had before, just make a whole bunch of allocations
3:06
happen, and the fact that these are lists and other things that can contain data
3:12
is actually important. OK, like this isn't enough.
3:14
You need stuff like this. Anyway,
3:16
what we're going to do is we're going to allocate a bunch of things,
3:20
and if there's enough allocations, potentially,
3:23
it could trigger the garbage collector to run.
3:25
So we may see cleanup before 3,
3:28
or maybe not. Let's find out.
3:31
Look at that. So over here we created the objects. We had originally 2
3:38
variables and then the links to each other,
3:40
so that was a total of 4 (2 and 2), and took away the variables down to 1
3:44
and 1, and then eventually after we did a whole ton of allocation, thousands of lists
3:51
and then multiplying that and right tons of stuff happening here,
3:55
enough allocation and memory pressure was put on the system
3:58
that it said "OK, OK,
3:59
OK, we need to slow up for a minute and run the garbage collector,
4:03
see if there's any garbage". It found that cycle that we were talking about, that
4:09
we created there to doomed objects and said
4:10
"you know what? This is a cycle.
4:12
It's out", threw it away before this line happened.
4:16
So it was the behavior of all of this allocation and tell you what line or
4:20
whatever. But somewhere in this looping around,
4:23
making a bunch of stuff happen triggered the GC to run,
4:26
we'll talk about what the scenarios are for that,
4:28
of course, And then it did a collection,
4:31
it found the cycle, it deleted them both and then we carried on right?
4:35
So if I, again, if I comment this out and I run it,
4:38
you'll see the doomed deleted at the end because it was at the end of the
4:41
program. But if I do a bunch of stuff,
4:44
it gets deleted along the way because eventually there's enough memory pressure to trigger the GC
4:48
to go look around. So, unlike reference
4:51
counting, GC is not deterministic. But it's not as important because it only applies to
4:58
objects that are put into a cycle.
5:01
And in fact, the only place objects that can contain other objects,
5:04
right, like a list and contain other things or a class can contain other things.
5:08
But a number, a string,
5:10
all those things they are always, always reference counted.
5:12
It's just the container objects, if you will.
5:15
We're going to talk a lot about the algorithm,
5:17
but this is the basic way in which it works.