Python Memory Management and Tips Transcripts
Chapter: Investigating memory usage
Lecture: Profiling in PyCharm
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Well, let's start our exploration. Jump over here into PyCharm, and we're gonna use the tools we have at hand.
0:08
There's some profiling built right into PyCharm and you'll see that we're gonna need to switch to some other stuff that's better eventually,
0:14
but let's see what we got right here. Now, first of all, I want to take this function pattern clinging to memory
0:21
thing that we did before. I'm gonna just use that, we're gonna explore that. We're gonna take this and just copy it down into chapter nine
0:29
and we'll call it "app_to_profile" or something like that. Now, over here, we're doing a couple of things that we're not gonna need
0:37
anymore. We don't want to have that size utility, right? We don't want to do this stuff because we're not writing out the
0:46
code, not right out the memory usage, based on process load or whatever we're calling it there, amount of memory used by the process.
0:53
Instead, we're actually gonna look directly at it, using the tools. So let's go down here,
1:03
clean this all up. And also, we only want our "greedy_main", not our other main, so we'll just delete that one. Alright, I could have done that before,
1:14
but I kind of wanted to show you, we're just going to take that one, and just remove the reporting that we had built into it.
1:20
And now what I want to do is I want to first set this to run, so I can use my hotkeys on it, and yep, it still works. But no more reporting,
1:29
right? Well, let's go and profile it. Notice there's a bunch of buttons up here. This one's run and this one's debug.
1:36
Debug is fantastic. But this one is the one we're looking for. We could profile it, so let's get some profiler information here. Alright, sweet. Look,
1:47
you can see where we're spending a lot of time, a lot of time on randrange, randint, some list comprehensions and so on. But that's not the best view.
1:56
Let's go zoom in over here and look at this. We got lucky and zoomed right into the part that we wanted. Over here,
2:04
you can see that our script is running, it's calling main, it's calling scale_data, load_data, and filter_data. And load_data,
2:12
well is taking the longest, right? It's taking two seconds. What about the memory usage? Hmm, nothing about the memory usage,
2:20
unfortunately. So this is only computational time, Nothing to do with memory, right? And a lot of the profiling tools,
2:28
are like that. They'll profile CPU time, but they won't profile the actual memory allocation that happens to be going on.
2:35
So this is cool, and it's really neat to see that filter_data and scale_ data were much faster than load_data.
2:43
But those other two, they do happen to use a lot of memory. We're gonna put this aside and say this is a great understanding of how our code
2:50
is running in terms of time, but we want to understand it in terms of memory. So we're gonna pull in some different tools to make that happen.