Python Jumpstart by Building 10 Apps Transcripts
Chapter: App 9: Real Estate Analysis App
Lecture: Concept: generator pipelines

Login or purchase this course to watch this video and the rest of the course contents.
0:00 For our final concept in this app, let's look at building data pipelines out of generators.
0:06 Now, these generators could be created from generator methods or generator expressions. So let's see how we might use this to answer
0:15 some questions about the housing market. Our first task is let's find all the houses that match my requirements,
0:21 and let's say personally I am looking for at least three bedrooms, New York city, any city will do for a first pass and so on.
0:28 Next, maybe we want to find all the houses that are for sale that I might in theory buy, so that houses that come out on the first set,
0:38 maybe those are some are for sale some aren't. Here we want the ones that are for sale,
0:42 and then, I'd like to take the ones that are for sale and then look around me. Ok, so I am going to start out with this the results of task one
0:50 and filter them further down and process them more in task two, and then maybe I'll process them still further in task three, so how do we do that?
0:59 Here we can get all the real estate transactions and use this data to start answering these questions, so I've got this is interesting method
1:06 and that was kind of like check for three bedrooms at least and so on, so some sort of criteria that I am looking for.
1:11 Now there is no guarantee that these transactions represent houses nearby, or that they are even for sale.
1:17 But then I can take the results of this generator, interesting tx, and pass them to another generator,
1:24 in this generator we look for the houses that are for sale, we don't care about all the houses that are for sale,
1:29 only the ones that I would be interested in. And because they are both generators,
1:33 they do this in a progressive as you consume the data they pull the data, they don't process it all at once, they don't load it in memory all at once,
1:41 all those sorts of things so we saw in application eight. Now, we can ask further questions,
1:46 we can say I would like the ones that are potentially saleable, which when I pull from that set it of course does its test
1:53 but to get those items it pulls back from all the transactions so there is kind of this pipeline or chain
1:58 depending on what which direction you think of it from, and we'll loop through that using yet another generator
2:03 and we'll say for all the ones that are for sale that are interesting to me, I would like to reduce that to ones just nearby,
2:10 maybe I don't want to move cities I just want to look at hey let's find all the houses I'd be interested in Portland Oregon, or wherever.
2:17 So, we can use these generators to build pipelines, you can take one generator and pass it to the next
2:23 and then take that generator and pass it to the next and it creates this really cool pipeline of processing that's extremely efficient
2:30 and it doesn't even require us to write methods.

Talk Python's Mastodon Michael Kennedy's Mastodon