Move from Excel to Python with Pandas Transcripts
Chapter: Data I/O (input and output)
Lecture: Pandas loves large files
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
so we saw how Excel failed to read in this customer transaction file. So let's see how Pandas does.
0:07
I'm gonna rerun my notebook and let's try reading in that CSB file, and it's done. Now let's take a look at the data and make sure it's
0:20
all there. Looks like the heads there. And this is nice so we can see how Maney Rose we have. Looks like all the data is there,
0:28
and we can see that it's 83 84 megabytes of data. Look at all the rows that we have and the data type.
0:36
So everything seemed to be read in really quickly. And one of the things you can do in Jupyter notebooks if you want to see
0:44
how long something takes, you can use the time it magic command and let's run
0:50
that. And what it does is it actually runs through the command multiple times and
0:54
kind of averages. The time it takes so in this example takes longer than normal
0:59
What's really nice about this is it takes just a little bit over one second to read that really large file that Excel couldn't handle,
1:09
and that really brings home. I hope for you how powerful pandas is and how it enables you to work with things that excel can't do.
1:19
So let's even continue. Continue this. What if we wanted to do just some really quick analysis?
1:26
Maybe we just want a quick group by and we can see how much we sold books, pencils and pens. And since it's a little hard to read,
1:37
lifts a plier styling to it just and now we've got nice dollar signs on it We can see $52 million and books $13 million pencils in $20 million in pens
1:48
all with a handful of pandas commands very quickly done in a way that we