Python Memory Management and Tips Transcripts
Chapter: Memory and functions
Lecture: Functions clinging to memory
0:00 Let's kick off this chapter by just diving right into some code.
0:03 So back to our demos. And over here in chapter seven,
0:10 we're gonna work on something about variable names,
0:13 something incredibly simple that's going to dramatically change the amount memory we're using. We'll call it
0:26 "clingy_functions", and the reason we're calling that is because
0:29 I want to give a little bit of credit to the guy who wrote an article
0:32 that talked about this pattern first.
0:36 Itamar wrote an article about functions clinging to memory,
0:40 how python function calls can increase your memory usage.
0:43 You can read it over here and you can listen actually to the interview I did
0:46 with him on Talk Python right there.
0:48 Now, this is not an exact copy of what he did,
0:50 but it's highly inspired. So thanks for that.
0:53 So we'll start out with our fmain,
0:55 as always. Now we're going to do three things here.
0:59 We're gonna come over and we're gonna have some data.
1:02 We're gonna call the data we start with "original", we're gonna say "load_data" like that,
1:07 and then we're gonna have some scaled data.
1:09 We're going to take it and
1:10 reprocess it, Maybe normalize it or something like that.
1:14 And we'll say "scale_data", pass in the original data.
1:18 We'll pass in this list or something like that,
1:20 and it's gonna generate a new list with updated pieces of information.
1:26 Okay, and It's not gonna change it,
1:27 It's gonna create a new one and we're gonna store it in "scaled",
1:29 and what do we want to do?
1:30 Maybe my favorite number approximately right,
1:33 2.718, e of course. And then we'll have "filtered".
1:38 We don't necessarily want to keep all of these.
1:40 Actually, I want to do, I'll do filtered
1:42 first. We'll do "filter_data(original)",
1:49 and this will be "filtered". And finally,
1:52 we'll just print out "Done!"
1:54 Okay, so what we're gonna need to do is we're going to need to write the
1:58 three functions: load_data, filter_data, and scale_data,
2:01 but the general idea is that we're doing this like pipeline of processing.
2:05 We're gonna load up some information and then we're gonna feed it on to the next step,
2:08 which is to filter it.
2:09 We only want to look at some of that data.
2:12 We're going to take the result of that step of this pipeline and feed it under
2:15 the next one and scale it.
2:17 And this can go on and on and on.
2:19 But, you know, I think three is going to be more than enough to
2:21 show what's happening here. So this is gonna be a really interesting use case,
2:26 and you'll see that we'll be able to dramatically change the memory usage here by just
2:30 making very small changes to this code in ways that you would probably get criticized for
2:36 if you were optimizing for readability.
2:38 But if you're not optimizing for readability,
2:40 but actually so the thing will run on the amount of hardware you have because you're
2:43 running out of memory or something like that,
2:45 all of a sudden, it feels pretty clever.
2:47 So that's what we're gonna do in this series of demos.