Python Memory Management and Tips Transcripts
Chapter: Memory and functions
Lecture: Functions clinging to memory
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Let's kick off this chapter by just diving right into some code.
0:03
So back to our demos. And over here in chapter seven,
0:10
we're gonna work on something about variable names,
0:13
something incredibly simple that's going to dramatically change the amount memory we're using. We'll call it
0:26
"clingy_functions", and the reason we're calling that is because
0:29
I want to give a little bit of credit to the guy who wrote an article
0:32
that talked about this pattern first.
0:36
Itamar wrote an article about functions clinging to memory,
0:40
how python function calls can increase your memory usage.
0:43
You can read it over here and you can listen actually to the interview I did
0:46
with him on Talk Python right there.
0:48
Now, this is not an exact copy of what he did,
0:50
but it's highly inspired. So thanks for that.
0:53
So we'll start out with our fmain,
0:55
as always. Now we're going to do three things here.
0:59
We're gonna come over and we're gonna have some data.
1:02
We're gonna call the data we start with "original", we're gonna say "load_data" like that,
1:07
and then we're gonna have some scaled data.
1:09
We're going to take it and
1:10
reprocess it, Maybe normalize it or something like that.
1:14
And we'll say "scale_data", pass in the original data.
1:18
We'll pass in this list or something like that,
1:20
and it's gonna generate a new list with updated pieces of information.
1:26
Okay, and It's not gonna change it,
1:27
It's gonna create a new one and we're gonna store it in "scaled",
1:29
and what do we want to do?
1:30
Maybe my favorite number approximately right,
1:33
2.718, e of course. And then we'll have "filtered".
1:38
We don't necessarily want to keep all of these.
1:40
Actually, I want to do, I'll do filtered
1:42
first. We'll do "filter_data(original)",
1:49
and this will be "filtered". And finally,
1:52
we'll just print out "Done!"
1:54
Okay, so what we're gonna need to do is we're going to need to write the
1:58
three functions: load_data, filter_data, and scale_data,
2:01
but the general idea is that we're doing this like pipeline of processing.
2:05
We're gonna load up some information and then we're gonna feed it on to the next step,
2:08
which is to filter it.
2:09
We only want to look at some of that data.
2:12
We're going to take the result of that step of this pipeline and feed it under
2:15
the next one and scale it.
2:17
And this can go on and on and on.
2:19
But, you know, I think three is going to be more than enough to
2:21
show what's happening here. So this is gonna be a really interesting use case,
2:26
and you'll see that we'll be able to dramatically change the memory usage here by just
2:30
making very small changes to this code in ways that you would probably get criticized for
2:36
if you were optimizing for readability.
2:38
But if you're not optimizing for readability,
2:40
but actually so the thing will run on the amount of hardware you have because you're
2:43
running out of memory or something like that,
2:45
all of a sudden, it feels pretty clever.
2:47
So that's what we're gonna do in this series of demos.