Python Memory Management and Tips Transcripts
Chapter: Memory and classes
Lecture: Where do classes store memory?
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's do another little exploration here. To understand why classes take as much memory as they
0:06
do, we need to figure out where they store their data. and it's also setting
0:11
the stage for a very big improvement that we're gonna make in a minute,
0:15
but you need to understand this first.
0:16
So let's go and say "app_storage". Set that that to run. Do our fmain magic, and
0:26
up here, I'm actually gonna paste some code
0:28
that's not worth watching me type out.
0:30
But, we're gonna do a little trick so
0:31
we can import size utility. I'm gonna define a class called "thing".
0:35
Now, thing contains thing1 and thing2,
0:38
and it has a way to print out what things
0:41
it has. So a thing object with the things in here, and also like this.
0:46
So we're just going to use that. There's not a whole lot to it,
0:49
just think of this as a standard class that has two fields,
0:53
and it just knows how to talk about itself,
0:54
Okay? That's all there is that's happening here.
0:57
So let's go and create some things. We'll have a thing, which is a hat, and a
1:05
mat, you know, thing1 and
1:07
thing2, we could do this Dr.Seuss style,
1:09
I suppose. Hat and mat, dog and cat,
1:15
bat and bird, car and bike.
1:21
And let's just print out the things, we'll make a new line. There
1:26
we have thing object at some address with hat and
1:29
mat. Notice 20, and 4, different location, with dog and cat and so on.
1:34
These are our things. Pretty cool,
1:36
right? Nothing special there. Standard class stuff.
1:40
If you look at any of these things,
1:43
any class at all, it will have a "__dict__"
1:46
and it's, this is where Python fields,
1:50
when you say "self.something" here,
1:53
what that really does is it creates an entry.
1:55
This is equivalent to saying
1:57
"self.dictionary of thing1 equals t1", right?
2:03
Those, the lines above these two are the same. Put that down here so I can
2:11
keep it. Those two things mean the same thing.
2:15
Alright, so whenever you have an instance of a class,
2:18
you have the instance of a dictionary on top of all the other stuff that might
2:22
have to be tracked and whatnot.
2:24
So the cost in terms of memory of a class is a dictionary.
2:30
Now you might think like many languages,
2:32
the cost is that string and that string,
2:36
but there's actually two sides to that coin.
2:38
The cost is this string right here, thing, and that. The entry that goes there,
2:45
these are actually reused. So there's only one,
2:49
I guess there's not too much,
2:50
but there is a pointer, got a 64 bit system, so that's 8 bytes plus this thing
2:54
for every field. So that's pretty interesting.
2:58
And down here, this is gonna be one of those.
3:02
So let's actually look at all of them.
3:06
So we'll say "obj.__dict__, for obj in things".
3:13
And then let's just print out the dictionaries. And look at that. Here we have a
3:20
thing with hat and mat and here we have a dictionary, thing1
3:23
is hat, thing2 is mat. thing1
3:26
is dog, thing2 is cat, right?
3:28
There it is. Those are the fields
3:31
self.thing1 self.thing2.
3:35
So you might wonder, well how much size does this use? So we could print
3:39
out the size, we'll just get the first one of the dictionary.
3:45
How much size does that use?
3:48
And let's also do class, things zero. Class itself. Perfect. So we run those.
4:01
You can see the size of the dictionary.
4:04
Ah, we're using the wrong one.
4:06
Let's go and use our size
4:09
util, get full size of the object.
4:11
That's right. It's not traversing,
4:13
is it? There we go.
4:15
That makes more sense.
4:16
So the dictionary is 318 and the class is 366.
4:20
So it's another 48 Bytes added on here to
4:24
have a class, but really the understanding that the dictionary is the thing that really
4:29
holds most of the stuff to do with the class.
4:32
That's pretty interesting, right? So this is where it's stored. Now,
4:36
question is, are these different dictionaries?
4:38
Are they copies? What's the story?
4:41
So what we can actually do is we can go and look at the locations of
4:45
the dictionaries. We could say "this is going to be the
4:47
id of d, for d in dicts".
4:51
And we could just print those, print the locations.
4:55
And If we look at that, what this is telling us is these are the memory
4:58
locations of the dictionaries of the classes. And so the same class,
5:03
over and over and over again,
5:04
right? Those are different. So what that's telling you is there's every time you say
5:08
a new class, you get a new dictionary and that allocates some bits that are
5:12
required to be tracked and managed in memory for every single one of these. In particular,
5:18
as I said at the beginning,
5:20
that you've got to keep track of the entries,
5:22
right? You can see thing1, thing1, thing1, now the value of that string
5:28
is actually reused, but, the fact that it's appearing over and over,
5:32
you still have to keep track of that in the dictionary,
5:34
right? And that takes a lot of space, relatively. Over here,
5:37
we've got new dictionaries. You might wonder,
5:40
why is that required? So let's go over here and say things, go to the last
5:47
thing, I don't know which one it is,
5:49
but we're going to say that,
5:50
um, it was thing1 and thing2,
5:53
those are its two fields. Let's say we want to give it a
5:55
thing3. Python is a dynamic language,
5:58
and will let you just dynamically apply these things. Now,
6:00
PyCharm says "you're asking for a hurt here",
6:04
but you can tell PyCharm
6:05
"don't bother me, I want to do this" and then we can go and we
6:09
can print out all the things again.
6:11
Let's do, I'll just print the dictionaries.
6:18
So check this out. We have different ones, and notice thing1
6:21
thing2, thing1
6:24
thing2, and the next one,
6:25
thing1 thing2, but the third
6:27
one, thing1 thing2
6:29
thing3, right? Now there's a key
6:31
3, thing3 in the third one because we did this line 59 here,
6:36
but the others they were unmodified.
6:38
Because of this dynamic nature, you can just go to an object and go "bam,
6:41
You have more stuff than you knew about".
6:43
That means you have to have a dictionary that's dedicated to each class that tracks all
6:47
the things that not just came with it,
6:49
but also were dynamically added. That's really flexible,
6:52
but it also adds a lot of overhead.
6:54
Keep that in mind. We're going to see if we could trade off this flexibility
6:59
for much better performance. Hint, we can. The take away from
7:03
this is whenever you have a standard class,
7:05
the fields actually get stored like this into "__dict__" field name equals field value,
7:12
and when you create a class,
7:14
each one of them gets a separate,
7:16
dedicated dictionary that is its own thing that was created in memory and managed and allocated
7:22
and populated and taken care of like that.
7:24
So we, come over here and we look for the locations.
7:27
They're all different. And the reason that you have a unique one for each class
7:30
is to support this dynamism here,
7:33
right? Thing3 is not really an aspect of things,
7:36
that's why PyCharm, if I don't tell it to stop, complains unresolved attribute reference
7:41
thing3 for class thing, because you're kind of not treating this right.
7:45
But it will accept it, and some programs were written this way,
7:48
so Python needs to be flexible for classes like this.