Python Memory Management and Tips Transcripts
Chapter: Recovering memory in Python
Lecture: GC in action
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's look at this cycle thing in the context of our little reference counting app that
0:05
we've done, and I'm just gonna print out of the top like this running reference
0:14
counting demo, cause we're gonna have another kind and these are gonna be super similar so let's go over here and just make a
0:20
copy, say, this is going to be "gc", and this is gonna be the "gc demo", and this is going to explicitly enable it. We don't have to do that,
0:29
but I explicitly disabled before so let's make sure of whatever else happened, It's in there. So we're gonna do kind of what I talked about in the
0:38
previous videos, we're gonna have 2 things, person 1, person 2, and We're gonna say "v1.friends.append(v2)", and vice versa,
0:48
right? These are 2 people who are friends. Let's go and tell it that this is alright. We're going to tell it "this is a list of friends".
1:00
There we go. Make sure it's a list. Now, we want to have the id. So let's have this "id1 and id2, v1 and v2" just
1:11
for the same reason we needed to keep track of those. So, id1, now let's change this to "counts are", and id2.
1:23
We don't need our step 2, well, we're gonna change what step 2 means. Let's go and actually set "v1 = none", and
1:32
"v2 = none", and then we'll put our reference count out again, like that, setting it to none twice is not gonna do anything.
1:44
Let's set it to be the end of the method and see when things get cleaned up. Maybe one of these will go away. Maybe both. We don't know.
1:52
Let's find out. So we're gonna run this one. Well, look at that. So we've created a new object, doomed1 and doomed2, those are Michael and Sara,
2:04
for example, and we had up here, I'll make this more legible for you, after this, we had one pointer to each one reference count,
2:13
now they're friends, so they pointed at each other, that incriminated it again, and you can see now the reference counts are 2 and
2:19
2, and we threw away the original variables and went back to 1 and 1 but they're never going to get better than that because of the cycle.
2:26
So we went all the way to the end of the method and they never got cleaned up. What happened? Well,
2:32
"gc" is non-deterministic. It's based on how the program behaved. We're gonna talk about that in a minute.
2:38
What are the gates that it has to go through and where the rules it uses to decide when to run, what objects to look at and so on?
2:46
But the short version is just cause these are free, the garbage collector is not constantly running around. It can only run so often,
2:53
otherwise things would be super inefficient. So let's go and make some stuff happen. Both to step 3. Down here
3:03
I'm gonna just, kind of like we had before, just make a whole bunch of allocations
3:07
happen, and the fact that these are lists and other things that can contain data is actually important. OK, like this isn't enough.
3:15
You need stuff like this. Anyway, what we're going to do is we're going to allocate a bunch of things, and if there's enough allocations, potentially,
3:24
it could trigger the garbage collector to run. So we may see cleanup before 3, or maybe not. Let's find out.
3:32
Look at that. So over here we created the objects. We had originally 2 variables and then the links to each other,
3:41
so that was a total of 4 (2 and 2), and took away the variables down to 1
3:45
and 1, and then eventually after we did a whole ton of allocation, thousands of lists and then multiplying that and right tons of stuff happening here,
3:56
enough allocation and memory pressure was put on the system that it said "OK, OK, OK, we need to slow up for a minute and run the garbage collector,
4:04
see if there's any garbage". It found that cycle that we were talking about, that we created there to doomed objects and said
4:11
"you know what? This is a cycle. It's out", threw it away before this line happened.
4:17
So it was the behavior of all of this allocation and tell you what line or whatever. But somewhere in this looping around,
4:24
making a bunch of stuff happen triggered the GC to run, we'll talk about what the scenarios are for that, of course, And then it did a collection,
4:32
it found the cycle, it deleted them both and then we carried on right? So if I, again, if I comment this out and I run it,
4:39
you'll see the doomed deleted at the end because it was at the end of the program. But if I do a bunch of stuff,
4:45
it gets deleted along the way because eventually there's enough memory pressure to trigger the GC to go look around. So, unlike reference
4:52
counting, GC is not deterministic. But it's not as important because it only applies to objects that are put into a cycle.
5:02
And in fact, the only place objects that can contain other objects, right, like a list and contain other things or a class can contain other things.
5:09
But a number, a string, all those things they are always, always reference counted. It's just the container objects, if you will.
5:16
We're going to talk a lot about the algorithm, but this is the basic way in which it works.