Move from Excel to Python with Pandas Transcripts
Chapter: Data I/O (input and output)
Lecture: Pandas loves large files
0:00 so we saw how Excel failed to read in this customer transaction file. So let's see how Pandas does.
0:07 I'm gonna rerun my notebook and let's try reading in that CSB file, and it's done. Now let's take a look at the data and make sure it's
0:20 all there. Looks like the heads there. And this is nice so we can see how Maney Rose we have. Looks like all the data is there,
0:28 and we can see that it's 83 84 megabytes of data. Look at all the rows that we have and the data type.
0:36 So everything seemed to be read in really quickly. And one of the things you can do in Jupiter notebooks if you want to see
0:44 how long something takes, you can use the time it magic command and let's run
0:50 that. And what it does is it actually runs through the command multiple times and
0:54 kind of averages. The time it takes so in this example takes longer than normal
0:59 What's really nice about this is it takes just a little bit over one second to read that really large file that Excel couldn't handle,
1:09 and that really brings home. I hope for you how powerful pandas is and how it enables you to work with things that excel can't do.
1:19 So let's even continue. Continue this. What if we wanted to do just some really quick analysis?
1:26 Maybe we just want a quick group by and we can see how much we sold books, pencils and pens. And since it's a little hard to read,
1:37 lifts a plier styling to it just and now we've got nice dollar signs on it We can see $52 million and books $13 million pencils in $20 million in pens
1:48 all with a handful of pandas commands very quickly done in a way that we