Python Memory Management and Tips Transcripts
Chapter: Allocating memory in Python
Lecture: Big objects may actually be many small ones
0:02 As you saw, in just the last video,
0:04 Python has this Small Object allocator.
0:07 Well, if small objects are treated differently than other objects,
0:12 you want to know what objects are
0:13 small and more of them are small than you would think,
0:17 because most big objects are actually just lots of small ones.
0:22 So let's take a look at one example.
0:23 There's many we could talk about.
0:24 So here we've got data,
0:26 a list that contains 50 integers, 1 to 50, and in a lot of languages,
0:32 the way this would work is you would have just one giant block of memory that
0:36 would be, say, 8 times
0:38 50 long for each 8 byte number that's gonna go in there 4 times that right
0:43 200 bytes if they're regular integers and so on.
0:46 And it's just one big thing.
0:48 But that's not how it works
0:49 in Python. When you put stuff together like this,
0:53 you're gonna end up with the list that has,
0:55 you know, a bunch of pointers as many pointers as there are in the actual
1:00 list. Plus probably a few extra for that buffering
1:03 so you don't reallocate like I talked about,
1:05 but what you're actually gonna do is have a bunch of little objects that are being
1:09 pointed to by the parts of the list.
1:11 And if you have a class,
1:12 you're gonna have fields in the class.
1:14 The things that are in there are like strings and numbers and maybe other lists those
1:19 are not part of that object in terms of how big it is,
1:22 those are outside of there, pointed to by,
1:25 pointed to by the variables within that data structure.
1:29 So these objects that feel like they're big, often they're many small ones,
1:34 and if that's the case, all of these little things get stuck into the algorithm
1:38 applied to the Small Object Allocator,
1:41 not a big object. Even though taken as a whole,
1:43 they might use tons of memory.
1:45 Most of the parts will probably still be effectively as far as pythons concerned
1:49 small objects.