Python Memory Management and Tips Transcripts
Chapter: Allocating memory in Python
Lecture: Big objects may actually be many small ones
Login or
purchase this course
to watch this video and the rest of the course contents.
0:02
As you saw, in just the last video,
0:04
Python has this Small Object allocator.
0:07
Well, if small objects are treated differently than other objects,
0:12
you want to know what objects are
0:13
small and more of them are small than you would think,
0:17
because most big objects are actually just lots of small ones.
0:22
So let's take a look at one example.
0:23
There's many we could talk about.
0:24
So here we've got data,
0:26
a list that contains 50 integers, 1 to 50, and in a lot of languages,
0:32
the way this would work is you would have just one giant block of memory that
0:36
would be, say, 8 times
0:38
50 long for each 8 byte number that's gonna go in there 4 times that right
0:43
200 bytes if they're regular integers and so on.
0:46
And it's just one big thing.
0:48
But that's not how it works
0:49
in Python. When you put stuff together like this,
0:53
you're gonna end up with the list that has,
0:55
you know, a bunch of pointers as many pointers as there are in the actual
1:00
list. Plus probably a few extra for that buffering
1:03
so you don't reallocate like I talked about,
1:05
but what you're actually gonna do is have a bunch of little objects that are being
1:09
pointed to by the parts of the list.
1:11
And if you have a class,
1:12
you're gonna have fields in the class.
1:14
The things that are in there are like strings and numbers and maybe other lists those
1:19
are not part of that object in terms of how big it is,
1:22
those are outside of there, pointed to by,
1:25
pointed to by the variables within that data structure.
1:29
So these objects that feel like they're big, often they're many small ones,
1:34
and if that's the case, all of these little things get stuck into the algorithm
1:38
applied to the Small Object Allocator,
1:41
not a big object. Even though taken as a whole,
1:43
they might use tons of memory.
1:45
Most of the parts will probably still be effectively as far as pythons concerned
1:49
small objects.