Python Memory Management and Tips Transcripts
Chapter: Investigating memory usage
Lecture: Profiling in PyCharm
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Well, let's start our exploration.
0:02
Jump over here into PyCharm,
0:03
and we're gonna use the tools we have at hand.
0:07
There's some profiling built right into PyCharm and you'll see that we're gonna need to
0:11
switch to some other stuff that's better eventually,
0:13
but let's see what we got right here.
0:15
Now, first of all, I want to take this function pattern clinging to memory
0:20
thing that we did before. I'm gonna just use that,
0:22
we're gonna explore that. We're gonna take this and just copy it down into chapter nine
0:28
and we'll call it "app_to_profile" or something like that.
0:34
Now, over here, we're doing a couple of things that we're not gonna need
0:36
anymore. We don't want to have that size utility,
0:41
right? We don't want to do this stuff because we're not writing out the
0:45
code, not right out the memory usage, based on process load or whatever we're calling
0:50
it there, amount of memory used by the process.
0:52
Instead, we're actually gonna look directly at it,
0:57
using the tools. So let's go down here,
1:02
clean this all up. And also, we only want our "greedy_main", not our other main, so
1:09
we'll just delete that one.
1:12
Alright, I could have done that before,
1:13
but I kind of wanted to show you, we're just going to take that one,
1:16
and just remove the reporting that we had built into it.
1:19
And now what I want to do is I want to first set this to run,
1:23
so I can use my hotkeys on it,
1:26
and yep, it still works. But no more reporting,
1:28
right? Well, let's go and profile it.
1:32
Notice there's a bunch of buttons up here.
1:33
This one's run and this one's debug.
1:35
Debug is fantastic. But this one is the one we're looking for.
1:40
We could profile it, so let's get some profiler information here.
1:45
Alright, sweet. Look,
1:46
you can see where we're spending a lot of time,
1:48
a lot of time on randrange, randint, some list
1:51
comprehensions and so on. But that's not the best view.
1:55
Let's go zoom in over here and look at this.
1:57
We got lucky and zoomed right into the part that we wanted. Over here,
2:03
you can see that our script is running, it's calling main, it's calling scale_data, load_data,
2:09
and filter_data. And load_data,
2:11
well is taking the longest, right? It's taking two seconds.
2:14
What about the memory usage? Hmm,
2:18
nothing about the memory usage,
2:19
unfortunately. So this is only computational time,
2:23
Nothing to do with memory, right?
2:25
And a lot of the profiling tools,
2:27
are like that. They'll profile CPU time,
2:29
but they won't profile the actual memory allocation that happens to be going on.
2:34
So this is cool, and it's really neat to see that filter_data and scale_
2:38
data were much faster than load_data.
2:42
But those other two, they do happen to use a lot of memory.
2:45
We're gonna put this aside and say this is a great understanding of how our code
2:49
is running in terms of time,
2:51
but we want to understand it in terms of memory.
2:53
So we're gonna pull in some different tools to make that happen.