Data Science Jumpstart with 10 Projects Transcripts
Chapter: Project 6: Working with Time Series - Air Quality over Time
Lecture: Checking for Missing Values and Shape in Pandas
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
I'm going to show you how to start cleaning up the data. Okay, so one of the things I might want to know is, are there missing values?
0:08
I'm going to sum these up across the columns, not the rows. And what will this give me? This will give me back a series
0:15
that should have the same index as this data frame, but it should give me the count of how many values are missing in each of those rows.
0:23
So let's run that. And you see that a lot of these are missing all of the values. Okay, let's look at the shape.
0:31
I'm going to use some pandas to filter a little bit. I'm going to say, I want you to return me the rows
0:37
where the number of values missing is less than 17. The total number in this is 9,471. It looks like we have 9,357.
0:49
So this filters out the rows where all of the values are missing.