RESTful and HTTP APIs in Pyramid Transcripts
Chapter: Your first service
Lecture: Adding data
0:02 So we have the web exchange part of our API created
0:05 this doesn't look like a whole bunch of auto data,
0:09 we need to add some data and that brings us to Kaggle.
0:13 Kaggle is a data science competition place,
0:16 they have many different data sets
0:19 so here you can see they've got this used car database
0:22 with over 370 thousand used cars scraped from Ebay Kleinanzeigen
0:28 so basically Ebay Germany, and it's got a bunch of German cars,
0:33 including all the Opels that we want to start selling
0:36 through our little dealership in our small town.
0:39 So what I've done is I've taken this, downloaded it,
0:42 and basically filtered it down just to the Opels
0:44 and then we're going to use that data in our API.
0:47 Now, there's no point in you watching me write this,
0:51 so I'm just going to paste a little bit of code here,
0:54 and this is somewhat big bit of code but let's close these all off,
0:58 now notice, we've got our csv here brand, name, price, year,
1:04 whether it has damage and when it was last seen for sale.
1:07 And notice they're all Opels because I filtered it down,
1:10 it doesn't have as many, there's still like tens of thousands of Opels but not as many,
1:15 and then we have this fake in memory database,
1:18 and basically it's using a dictionary
1:20 with the id of the car which it generates at runtime
1:23 when it first load s it up, as well as loading in this the file
1:28 so here you can see it calls load data, and the load data just pulls those in
1:32 creates a little temporary primary key thing that we can exchange in our API
1:37 and stores that car, it's important to note that we're using the dict reader
1:41 so the things here, this row that it represents an individual car,
1:47 that is a dictionary, that's super important because that means
1:50 we can serialize it just by straight returning it from our APIs,
1:53 if it weren't the case, then we would run into issues;
1:55 We'll switch to better a real database with real database entities
1:59 and we'll see to address that, but for now these things get loaded up
2:02 and returned as dictionaries so that helps us a lot.
2:05 So that's really all I want to talk about this,
2:08 we're going to come back into a proper database part later,
2:11 but we're just going to look at the two functions get all cars and get a car by id
2:15 car ids are strings, which is handy
2:17 because that means that no conversion is necessary
2:19 that's the way they come off our API or of the url.
2:22 Okay, so let's go over here and do this first part here,
2:26 let's go over here and say we want to get some cars
2:28 and it's going to be a repository,
2:30 we can import that, thank you PyCharm and we can call all cars
2:33 and notice that there is a limit, so we want to set the limit
2:36 to 25 cars or something to that effect,
2:39 so what we're going to get back is a list of car dictionaries
2:42 that are loaded up there and we can just return those cars, ok.
2:46 So in a real service, where you have ten thousand cars or twenty thousand cars
2:51 however many are in that csv file, you don't want to return all of them
2:55 we'd want to implement some sort of paging,
2:57 so maybe we'll come back and add paging,
3:00 you could tweak the url to basically pass like a skip level and a page size level
3:05 or you could hard code the pages size
3:07 and just do a page number to implement some kind of paging
3:10 but we probably don't want to return like everything
3:13 so we're limiting it to just the first 25 responses.
3:16 So let's try the all autos bit, okay it still runs, that's encouraging
3:21 so we can come over here and if I click on all autos
3:24 it shouldn't be that little test data, instead what do we get- oh that looks real
3:29 see Opel, Opel, Opel, and this is a little parser display thing that Firefox just added,
3:34 let's go look at the raw data, that's not so pretty, it's pretty printed
3:37 all right so check this out, we've got all of our Opels down here
3:41 I can bet if you count them there's 25 of them
3:44 you can see the brand, you can see the name,
3:46 this is basically what was typed into Ebay, what year,
3:49 how much they're asking in euros and importantly the id right here.
3:54 So this is great, the next thing that we want to implement
3:57 is going to autos/ that id and now it just say it is an auto,
4:05 well it told us that function was called but it doesn't tell us anything that's happening,
4:09 so this was pretty straightforward, right like all we had to do
4:12 is go to the database and say give me the cars and return it,
4:16 and tell it hey, you serialize that, you return that as json.
4:20 Now, we got lucky that that is json sterilizable,
4:23 otherwise it would have crashed, we would have to do some work
4:26 but it's not insurmountable, we can totally fix that.
4:28 This one takes a little bit more work,
4:30 okay so this one is nice, but I'm supposed to get that car id
4:34 and I can't just do car id that would be nice, Pyramid guys don't do that
4:39 so the way we're going to get the car ids we have to use the request
4:42 so we'll go to request, and then there's a couple of things that we can get here;
4:52 here we can basically do a type annotation,
4:55 and now you can see all the stuff that shows up here,
4:58 this works because it's Python 3, right.
5:00 So I can come over here and I can say well it's got a get,
5:03 notice this is a dictionary, it's got a post
5:08 which is another dictionary that has the post data,
5:11 get actually is not the url but this is the query string
5:15 but it also has one called match dict, which is not obvious that that's what you want
5:20 but match dict is where those little route cutout pieces of data go,
5:24 so this is a dictionary, so we can call get on it, and the thing we want is car id,
5:28 because that's what we called it in our route ok.
5:31 So, then we can come down and we can go to our repository
5:34 and we can say car=repository
5:39 and it happens to have a get car by id, that's not surprising, is it;
5:43 so we'll pass the car id, now let's just return the car
5:46 when we make changes to the Python code, we have to re-execute the app
5:51 so we restarted there, now this is not going to do what we hoped,
5:55 those ids were randomly generated in our little temporary database,
5:59 it will be permanent in sqlalchemy, but right now they're temporary,
6:02 so restarting the app means that probably comes up as nothing,
6:05 so if we look at the raw data, it's no, sad face.
6:08 This is not the desired behavior,
6:11 we definitely don't want to use null as a return value,
6:14 I'm going to store that there for a second, let's go over and refresh this,
6:17 so that we can get an id, let's go and get this one here
6:20 and it's going to give us our Opel,
6:26 but we're going to get Opel Signum 1.3 cdti,
6:29 all right so let's go over here, put that in,
6:33 now this because this exists in our database
6:36 should come back and boom, there it is, so we took out the raw data pretty printed
6:40 here is our nice little Opel, and 4700 euros,
6:44 maybe it's in good shape I don't know, I'm not sure if it's a good deal,
6:47 but there is, it has no damage, that's a good sign.
6:50 Ok, so now we kind of have our API working
6:53 but as we saw right here, this is not very API friendly,
6:58 we're using the http verbs, we're not using the status codes
7:03 to communicate that something went wrong, so let's go back and fix this up.
7:08 So we'll come over here and we'll say if not car,
7:12 so the way the database works and the way sqlalchemy or a mongo engine
7:17 or lots of things would work if I asked for a thing that's not there,
7:19 it's going to return none,
7:21 I could have my repository layer raise an exception
7:23 but it doesn't, it just returns none and flows that through.
7:26 So we're going to come down here and say if there's no car returned
7:29 then we want to somehow send a 404,
7:31 now we could set the headers and all sorts of funkiness
7:34 but the thing we really want here is a response,
7:36 we want to import that from pyramid response
7:40 and notice it takes a couple of things, we can set the body, not interesting
7:44 we can set the status, let's set the status to 404, okay;
7:48 now by the way, if I go back over here and I look at the headers
7:51 the response headers are, yeah, I don't see where it says
7:55 whether it's 200 or not but you can bet, it is a 200, okay, even though it's empty.
8:00 So we're going to say this time 404, and what else we want to set,
8:04 is we want to set the json body, so here we can pass a dictionary
8:07 that has the error, let's say there's an error and this can be the message,
8:12 so we'll create a message,
8:14 it's going to be the car with id such and such was not found
8:18 and in case they don't pass one, let's put a little back tick like that,
8:22 so the car id and creating the response is not enough, we have to return it.
8:29 So if we don't find a car in the database,
8:32 we're going to embrace the http response 404,
8:36 and we can still send a body by sending
8:38 some kind of model that represents the error
8:41 so be sure to rerun, refresh,
8:44 boom- error, the car with id such and such was not found.
8:48 Now if we go over here and look at this, we still don't see the 404,
8:51 I'll show you in a little bit where we can find that,
8:54 we'll use something other than the browser to pull that up.
8:57 But if we go over to the raw data, you can see now we've got this error.
9:02 I might be able to do it if I go to the network,
9:05 here we go, perfect, status 404, method equals get not found
9:10 and even though it's a 404 not found, we're still returning a nice message
9:20 we have a message that theoretically we could show to the user
9:23 in the front end, something like that.
9:26 So there you have it, we have our two methods written,
9:28 we don't have our pretty print json we just have the standard minified json renderer
9:32 but that's ok, we've got our thing that does all the autos
9:35 and the basic sort of hard coded limit and given an id,
9:39 we can go back here and we can call this function
9:43 by passing / api /auto/id and get an individual car, or not,
9:48 and get a proper http friendly response or 404 with the error message right there.