Python Memory Management and Tips Transcripts
Chapter: Efficient data structures
Lecture: Container sizes, arrays
0:00 Sp lists seem like a clear winner,
0:02 maybe, but remember, lists actually are super inefficient.
0:05 They've got their contiguous structure to hold the pointers,
0:09 but then they point out at pieces of numbers in memory.
0:14 And remember, those numbers are like 28 bytes a piece,
0:17 where storing most these numbers, they're ages,
0:20 they're not very big at all,
0:22 so we should be able to do better.
0:23 And if these numbers were larger,
0:25 still, remember, we're working with numbers that are within the range of that flyweight pattern.
0:29 So it's probably not as big of a deal as
0:33 It could be. I'll maybe tweak that just a little bit to see what we
0:36 get in terms of if it were different types of data.
0:40 But what we do is we're going to say we're "storing them as arrays".
0:45 So this is not a common type used in Python,
0:48 but it is an interesting one.
0:50 So we'll say "ar = array".
0:53 Now, this is a type that we can import like this, and, array,
0:58 and then you can give it a type. See that type code?
1:02 It can only hold one of the same things.
1:05 If we want it to hold integers,
1:07 we can use a "B". It can
1:08 also hold floats and stuff like that. We'll say "ar.fromlist" and we'll give it
1:14 the ages, let's grab this and put that here say "array_size =
1:29 size_util.get_full_size(ar)". "array_size".
1:37 Perfect. Now let's run it and look at that.
1:45 Wow. So instead of using 800 kilobytes or something way more for whatever that is
1:50 it's using 100. So there you can
1:54 see, way, way better. 106 kilobytes instead of 800.
1:58 So again, we're getting almost this 10x difference by choosing a slightly different
2:03 data structure. So that's pretty awesome.
2:08 If you go look at the documentation for array,
2:10 you'll see that there's different types like we're using a "B",
2:12 which is an unsigned character. So maximum size in bytes is 1,
2:17 so I think this is 0 to 256 probably,
2:19 but if you have longer data,
2:21 bigger data rather, like a float,
2:24 you can say it's an "F",
2:26 or you could say it's a "signed long".
2:28 But be aware that when you create one of these and you put something in there
2:32 every entry that we had before was just one byte,
2:36 and that was why it was really efficient.
2:38 Where as over here, if you say it's a float,
2:41 right, it's gonna take 4 times as much memory,
2:43 but of course it can hold the things that go in it.
2:46 So keep in mind, this is only for numbers.
2:48 We can't apply this cool trick to strings,
2:51 but nonetheless we still got cool memory reduction accomplished,
2:56 right? So we could drop that one down by 700 KB at least and have names
3:00 and then this type of array to store.
3:02 So if you're storing a lot of numbers,
3:04 it's very likely that these arrays are going to help you a lot.