Python Memory Management and Tips Transcripts
Chapter: Memory and functions
Lecture: Concept: Generators
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
We saw how incredibly easy it was to convert these functions that turned everything to a
0:04
list all at once and then handed them around over to generators.
0:09
Square brackets, parentheses. Done. Now again,
0:13
the consumption side, how you use it,
0:15
you have to be more aware of only going through that list once,
0:19
but if that's the use case you have anyway,
0:21
this is an incredibly easy way to do it.
0:24
We could also go and be more explicit for scenarios where it's not just a list
0:28
comprehension but you have more interesting code. Like this one
0:31
we converted using the yield keyword, and surprisingly,
0:35
it actually got shorter even though it sort of had more syntax that we had
0:39
to do. And it just blew the other patterns we had out of the water.
0:44
So this one, the memory growth in the example I had originally was 102
0:50
megabytes, and this one is zero. Zero!
0:56
Incredibly, it's because we only loaded like a handful of numbers at a
1:01
time, and that's it, and nothing else.
1:03
It's so awesome. This one,
1:05
on the other hand, I timed it outside of, I just threw some timing on the
1:09
demo so I'll check that into the source control,
1:11
This one takes about 1.11 seconds on my computer while I'm doing screen
1:16
recording, all that kind of stuff.
1:18
This one takes a little bit longer, 1.77 seconds.
1:22
So your paying a little bit of computational time for in 400X improvement in memory,
1:29
415 times better memory storing. That's not always worthwhile.
1:32
Maybe you have more memory, and it's just about getting things done quicker.
1:35
But if this is the case,
1:37
if you're under memory pressure, this is a fantastic pattern.
1:41
You should absolutely use these in data pipeline type of scenarios. It's really,
1:45
really powerful.