Python Memory Management and Tips Transcripts
Chapter: Memory and functions
Lecture: Concept: Generators
0:00 We saw how incredibly easy it was to convert these functions that turned everything to a
0:04 list all at once and then handed them around over to generators.
0:09 Square brackets, parentheses. Done. Now again,
0:13 the consumption side, how you use it,
0:15 you have to be more aware of only going through that list once,
0:19 but if that's the use case you have anyway,
0:21 this is an incredibly easy way to do it.
0:24 We could also go and be more explicit for scenarios where it's not just a list
0:28 comprehension but you have more interesting code. Like this one
0:31 we converted using the yield keyword, and surprisingly,
0:35 it actually got shorter even though it sort of had more syntax that we had
0:39 to do. And it just blew the other patterns we had out of the water.
0:44 So this one, the memory growth in the example I had originally was 102
0:50 megabytes, and this one is zero. Zero!
0:56 Incredibly, it's because we only loaded like a handful of numbers at a
1:01 time, and that's it, and nothing else.
1:03 It's so awesome. This one,
1:05 on the other hand, I timed it outside of, I just threw some timing on the
1:09 demo so I'll check that into the source control,
1:11 This one takes about 1.11 seconds on my computer while I'm doing screen
1:16 recording, all that kind of stuff.
1:18 This one takes a little bit longer, 1.77 seconds.
1:22 So your paying a little bit of computational time for in 400X improvement in memory,
1:29 415 times better memory storing. That's not always worthwhile.
1:32 Maybe you have more memory, and it's just about getting things done quicker.
1:35 But if this is the case,
1:37 if you're under memory pressure, this is a fantastic pattern.
1:41 You should absolutely use these in data pipeline type of scenarios. It's really,
1:45 really powerful.