Consuming HTTP Services in Python Transcripts
Chapter: Reading JSON data with requests
Lecture: Consuming GitHub JSON API
0:00 I bet by now you are ready to actually consume some real services,
0:03 we've made some requests and we played with Json,
0:07 but we haven't really put it all together
0:09 and we are going to start by doing that with github,
0:12 so github is a public API and we can use request issue a get to it
0:16 and then what we'll come back is some kind of Json document
0:18 that we can work with in our code,
0:22 so let's switch over to PyCharm and do that now.
0:24 So, I've made a new file here first to work with and let's go ahead
0:27 and just set that as he run configuration,
0:30 notice we are running out of our virtual environment and we'll just start by adding a little structure,
0:34 so we are going to import requests and then let's have a main method,
0:39 and let's conditionally call this, we can use PyCharm's little template
0:42 to help us out there, there are live template.
0:45 Okay, so we are going to need to get the url here for the API
0:49 and then of course we are going to do something like this,
0:52 maybe we are going to need to add some headers,
0:55 maybe we are going to need to do some other things,
0:57 but we are going to need to start by knowing what is the API we are going to actually work with.
1:02 So let's drop in on developer.github,com and check out through API
1:05 so if you scroll down there is a little bit of a getting started guide,
1:07 authentication, and so on, we are just going to go over here
1:10 and let's suppose we want to work with repositories,
1:13 so if we want to get information about like say repository here,
1:17 we could go to twitter bootstrap/bootstrap,
1:19 we could also go to the github repository for this project,
1:23 so we are going to access https://api.github.com/repos/ let's get the repo for this course,
1:31 so it would be mikeyckennedy that's the username
1:34 and then it's going to be the repo name, so over here, let me paste that,
1:38 okay so now we have our url, we are going to go to the repo
1:42 which basically contains well, literally this source code, that is kind of meta isn't it,
1:45 and we are going to go get our own thing here, now the question is what are we going to get back,
1:50 we can start by exploring this in a real simple way, we already know how to do this request here
1:55 and we can just print out the response.text okay, so this is kind of the lowest level,
2:00 let's go ahead and run that and see if this works,
2:03 cool, it looks like we've got something here, now I don't know about you,
2:06 this is a little hard to read, if you look at this scroll bar it's huge okay,
2:10 so that's not great, we technically could do this, import json.loads
2:18 and we can give it this, and we could then say json.dumps,
2:22 this might seem a little bizzare but then of course we could say indent=true,
2:25 run this and we get at least formated Json,
2:29 but, let's have a better way to explore this, shall we?
2:32 So let's jump over here and run postman.
2:35 Postman is great for exploring APIs, see I've got some saved ones,
2:39 that we are going to work with later for more complex interaction
2:42 when we get to modifying data on interesting APIs and so on,
2:46 but we can just put this url in here and notice, we can do a get put post
2:50 but this is just a get and we could set like headers,
2:53 for authentication if we needed it but we'll just go and send this off,
2:57 here we get what came back, so you can see we have information
3:00 about the full name of the repository, its id, the number of teams,
3:04 the number of forks, all sorts of stuff whether or not it has issues and so on,
3:08 okay, so we could print out the get url, the clone url, so let's do this,
3:13 let's say what we want to do is given the name of a repository and the user,
3:17 we would like to go and actually get the clone command in order to clone it,
3:22 or maybe get the url to open its documentation or homepage, okay?
3:26 So we are going to be doing that, by going and downloading this,
3:29 converting to Json and then let's say access the clone url.
3:33 Okay, so we've seen we've already gotten the text and we can also parse it like this,
3:38 but of course we should really check that this works, right,
3:41 and let's also get some input here so we'll do this.
3:47 So we'll write a little function here called get_repo_info
3:53 and we'll write that down here above the dunder main check,
3:56 and this is going to get two things, this is going to get the user, it's going to be input,
4:02 we'll just ask whoever is typing here, what is the username,
4:05 and then we'll do the same for the repo, and then we'll return user, repo.
4:14 Okay, great, so I am going to put this in a comment up here so we always have it,
4:19 in case you forget what the name especially this right here.
4:23 Alright, so we are going to do a little bit of formatting like so,
4:29 okay so that again is going to get what we need,
4:35 let's format that a little bit and tell PyCharm that no, my name is not misspelled.
4:40 Alright, so let's run it make sure this little step is working, what is username,
4:43 what is a repo name, this is going to be a copy paste,
4:47 and it looks like it still works, great. So the next step is to actually check,
4:52 because it could have gotten a 404 all sorts of things.
4:56 So, we'll do this, we'll say if resp.status_code != 200
5:01 then we'll print error accessing repo and then we'll just bale,
5:11 so if we put in stuff that's wrong, error accessing repo 404,
5:17 if we put in stuff that's right, everything is working, okay.
5:22 So this is great, we've got our information,
5:26 and maybe we should check that the format is Json,
5:29 but we are more or less guaranteed through the API that we are going to get Json,
5:34 so we could avoid this whole step and we can actually avoid
5:37 even using the Json module whatsoever, because requests actually already supports this,
5:44 so what we are going to do is we are going to have a repo data
5:48 and we are just going to go to the response
5:51 and if we know that the inbound data is Json,
5:54 we can just say Json and that will actually convert this to a Python dictionary
5:59 so if I just print the type of repo data and we run this,
6:07 and we put in something legitimate, we get a dictionary.
6:12 So the final thing, what we were trying to do the whole time
6:16 is we were going to say this,
6:19 print to clone this person's repo named let's just put a little bit info here,
6:25 so we'll say user repo print the command is get clone
6:34 and then whatever the clone url is here, so we'll say format
6:37 and let's just store this for a moment, so what we need to do to get this
6:41 if we go back over to postman, is we need to go to the dictionary
6:47 and get the thing called clone url, so we can just go to the repo data
6:52 and say either like this, this is a little bit dangerous
6:57 because who knows what you are going to get, you might get a key error,
7:00 so we could do a get and this will just return nothing, if it doesn't exist,
7:02 we'll say error no data, something like that,
7:08 so we are going to get this clone url, we can get rid of this
7:11 and we are going to come down here and just say here is the clone url to go.
7:14 Alright, so let's put this all together, we are going to run this username is mine,
7:18 repo name is the repo from this class consuming services python demos
7:23 and if I hit go, the clone command is get clone such and such.
7:29 Well, let's see if that is going to work.
7:38 Get clone that, oh it looks like it works.
7:43 I guess our service is working perfectly, so that is how we consume services that are Json based,
7:49 HTTP services at least read only get pay services with requests,
7:54 we just do basically construct the url we do a get
7:57 we make sure the status code is whatever we expect,
8:01 200 is not always what is expected,
8:05 sometimes if we were doing a post and creating thing,
8:07 201 would be expected but you check for what is the success code if you will,
8:11 and then we can call just the Json function right on the response
8:15 and it converts that straight to a dictionary, it's a not a huge step to save,
8:18 that we are skipping over using the Json module,
8:21 but it's just one less thing to think about
8:23 how do you work with Json from this other thing,
8:26 we just work with it straight from requests and then,
8:28 once you have it as a dictionary in memory,
8:31 well, it's no longer a web service it's a problem, it's just data in Python, go.