RESTful and HTTP APIs in Pyramid Transcripts
Chapter: Logging and monitoring
Lecture: Why you want monitoring and logging
0:01 In this chapter we're going to add
0:03 both logging and error monitoring to our web api.
0:07 First of all you might ask why do we care about monitoring, right,
0:10 just our clients let us know, yeah they probably will,
0:13 but there's a couple of things we kind of want to look for;
0:16 we want to have extra information
0:18 so that we can be sure we know what happened,
0:20 and we like to get notified before people start complaining.
0:24 This is a pretty common thing on the internet,
0:26 ooh our web application is crashed,
0:29 boom 500 sorry and it's cool to have like a fun error page
0:32 so at least it doesn't feel so bad, right,
0:34 but when our application crashes,
0:36 we want to have all the information that led up to this,
0:39 what browser were they using, what exactly set of steps did they take
0:43 what was the url, was there a query string, where they logged in
0:47 who were they longed in as, all these things are super important,
0:49 so having that in the log file is great, getting that notification exactly
0:53 when this error happens like maybe to our Slack channel
0:55 or to our email would be even better.
0:58 So, eventually when the user comes back in contact us
1:01 about the error and says hey I had this problem,
1:03 we can say yeah we're already researching it
1:06 we know what happened, we're going to fix it shortly.
1:08 That's the perfect answer if something goes wrong, right.
1:10 So, in this chapter we'll see how to do that.
1:12 All right, first of all let's look at the log file,
1:15 so here's the log file from Talk Python training
1:17 and there's a number of interesting things to see about it
1:21 I've actually cleaned up some of the data
1:24 so that we don't have like personal email addresses
1:28 or the other things like that that maybe shouldn't appear in a public course,
1:30 but just pulling up this log file shows us some really interesting things.
1:36 So, for example, we can see user actions, like here our user is logged in,
1:39 here's another user action, somebody subscribed to the mailing list,
1:43 and here's another, this person has logged out at this ip address.
1:47 So this is standard application operations, right
1:51 and we issue this to the log files a notice
1:53 like these are kind of significant events in the application life cycle,
1:56 but they're not highly important, right they're just stuff going on.
2:01 Now, we also have other things that maybe you wouldn't have expected
2:04 search engines, so down here you can see the search engines are trying
2:07 to dig into the actual course videos themselves,
2:10 they shouldn't be doing that, I don't know why they're trying to go here
2:14 I don't really care but you can see
2:17 that there is some four or four type stuff happening
2:19 around googlebot or something compatible with googlebot,
2:22 but I'm not sure what the heck that is;
2:24 there's also other interesting stuff that shows up,
2:26 for example here's somebody trying to hack the website
2:28 so they're looking for the wp login php page
2:33 which is what wordpress uses as its backend
2:36 to give you an admin access to the site
2:38 as you probably know, this site is built with Python, IT's a custom site
2:41 there is no wp login so those hackers can just take off right,
2:44 nothing really important happened here,
2:47 but it's interesting to know that someone was actually trying to hack us
2:50 in fact this happens all the time, it's happening here, and also down here.
2:53 So the log file lets us know what's going on in our application
2:57 and we'll even discover certain things like why are the search engines
3:02 trying to get to the private videos, I don't know,
3:05 but having a log file turns out to be really helpful
3:07 just in understanding how your apps are being used as well.
3:10 So this equally applies to web applications as it does to web apis.
3:14 Now, if something goes wrong, we want to know about it as soon as possible
3:18 so we'll briefly talk about setting up Rollbar
3:22 which is what I use for my web applications
3:25 you can really easily integrate Rollbar into Pyramid
3:28 and it will automatically capture all these errors and send them your way.