Move from Excel to Python with Pandas Transcripts
Chapter: Data I/O (input and output)
Lecture: Pandas loves large files
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
so we saw how Excel failed to read in this customer transaction file.
0:05
So let's see how Pandas does.
0:06
I'm gonna rerun my notebook and let's try reading in that CSB file,
0:15
and it's done. Now let's take a look at the data and make sure it's
0:19
all there. Looks like the heads there.
0:21
And this is nice so we can see how Maney Rose we have.
0:25
Looks like all the data is there,
0:27
and we can see that it's 83 84 megabytes of data.
0:32
Look at all the rows that we have and the data type.
0:35
So everything seemed to be read in really quickly.
0:38
And one of the things you can do in Jupiter notebooks if you want to see
0:43
how long something takes, you can use the time it magic command and let's run
0:49
that. And what it does is it actually runs through the command multiple times and
0:53
kind of averages. The time it takes so in this example takes longer than normal
0:58
What's really nice about this is it takes just a little bit over one second
1:05
to read that really large file that Excel couldn't handle,
1:08
and that really brings home. I hope for you how powerful pandas is and how
1:14
it enables you to work with things that excel can't do.
1:18
So let's even continue. Continue this.
1:20
What if we wanted to do just some really quick analysis?
1:25
Maybe we just want a quick group by and we can see how much we sold
1:31
books, pencils and pens. And since it's a little hard to read,
1:36
lifts a plier styling to it just and now we've got nice dollar signs on it
1:42
We can see $52 million and books $13 million pencils in $20 million in pens
1:47
all with a handful of pandas commands very quickly done in a way that we