Python for Entrepreneurs Transcripts
Chapter: Monitoring and logging in production
Lecture: Integrating logbook into our app
0:02 We've set up the structure of where we are going to store our information,
0:05 read it in from the config file and whatnot, let's use Logbook now.
0:08 So, this is probably no surprise, we'll start by saying "import logbook".
0:13 Now, remember, this needs to be put into our setup requirements,
0:18 this needs to be literally installed because we don't have it yet,
0:21 so first, let's install the package, thank you PyCharm, ta-da, OK,
0:28 I got to move away for a second, and then come up here,
0:32 hit it again and I can add it to the setup, and then it's saying we're not using it
0:36 but that's cool because we are going to that now.
0:39 So the first thing that we want to think about is where do we want the logs to go,
0:43 do we want them just to go to the console,
0:47 do we want them to be sent to a file? Things like that,
0:51 and my perspective here is in production, we want them to go to a file
0:55 and in development we just want them to go to the console,
0:58 so we'll set that up now.
1:01 Now this level when it comes in here
1:04 is going to be a text string straight out of the config file,
1:08 Logbook has its own enumeration, which represents this,
1:12 so I am going to make this very clear and say log_level_text,
1:15 we are going to need a function to convert that to Logbook levels,
1:19 so let me go and just paste that function in because once you see it, you'll be like
1:23 "wow", there is not point watch me write that,
1:26 so here what we are going to do is we are just going to have a function,
1:29 pass it a string, and it's going to come in and it's going to pull out logbook.CRITICAL,
1:33 if it sees the string CRITICAL, Logbook.ERROR, if it sees the string ERROR and so on.
1:38 And we are also stripping it and uppercasing it,
1:41 so there is a little bit of non equalization going on.
1:45 OK, so the first thing we are going to do is say "level...",
1:50 so we want to come here and say get the Logbook level
1:56 and we are going to pass in that text and it's going to convert it to that enumeration.
1:59 The next thing we got to do is decide where do we go,
2:04 where do we pass this information,
2:07 we could do a couple of things, we could queue this on the filename,
2:10 or we could also pass the development mode or whatever,
2:13 I am going to basically say if you supply filename, it's going into the file,
2:16 if you don't supply filename, it's going to go to the console
2:19 so we'll say something like this,
2:21 "if filename", now let's start with "not", so standard logging, right,
2:25 so if it doesn't go to the filename, we are going to say Logbook
2:29 and we are going to setup a stream handler,
2:32 and the stream we want to go to is standard out,
2:35 which we can get from sys, we'll say standard out, so that's just console,
2:39 and we want to set the level to level,
2:44 so remember, if I log in informational message,
2:47 but I say I only want to see streamed to the standard out, stuff that's warnings and errors,
2:54 that message will basically silently go away, which is really nice
2:57 because that lets us turn up and down the logging just in one place
3:00 and the rest of the app doesn't have to worry about it, right,
3:03 they log at whatever levels they think make sense, and then here we just adjust it.
3:07 And so we can set this up and then we need to push this to the application,
3:12 so basically that means make this globally available
3:15 as a stream handler for the rest of the application.
3:20 So later anywhere in our app we can just create a Logbook,
3:23 set some values and start going, and it will use this stream handler.
3:28 So we'll come down here and say if it's not the case we have no file,
3:31 we do actually want to go to the file system.
3:35 Now, we could just go straight and log to a file, but on busy application
3:40 that turns out to be a huge problem, like on Talk Python To Me,
3:44 recently I went and checked my logs, I had something like 3 GB of log text,
3:50 that's a serious amount of logging, right,
3:53 and you want to clean that out every now and then,
3:56 if this is one huge 3 GB text file, that's going to be a problem, trust me.
4:00 So what we are going to do is we are going to set this not just to a file
4:03 but we set to a timed rotation file handler, so basically we get a log file every day,
4:09 the base name plus the date.
4:13 Again, we are going to set the level, and then we are going to set a date format,
4:21 let me format it like this,
4:26 so we are going to print it out,
4:29 store the date in year-month-day format and again,
4:33 in this case we want to push this to the application,
4:36 so nowhere outside of this little block of code,
4:40 do we care or even know about how the logs are written;
4:46 we'll say here if you don't provide us a file,
4:49 we're just going to stream it to the standard out, but if you do provide us a file,
4:52 we are going to create a rotating file based on that base filename that you provided us.
4:57 Alright, so that's all good.
5:00 Now that this is in place, we can go over here and write this little startup log,
5:05 so when we go and try to access to the startup log,
5:07 we are going to return an instance of a Logbook,
5:09 and that is going to be of course leveraging this
5:12 but we don't have to directly say anything, so we are going to create a new Logbook,
5:16 and the thing that you actually create is a logger and in here you give it a name
5:21 and so this is like the area of your application, what part of your app is doing this,
5:27 so this remember is going to be used sort of through the startup process
5:30 and nowhere else so I am going to call this,
5:33 I'm just going to call it "App", right, this is the application startup level.
5:37 Let's go back up here one more time, and let's use our little startup log
5:40 that we are creating, and just print a message to say "hey, everything is working",
5:46 so we'll say the message is..., so we'll say logging initialize with the level
5:53 and the mode, whether or not it's in file mode or standard out mode.
5:59 OK, and then, we're going to say LogService, let's make that static,
6:07 actually let me change that to a static method.
6:15 Now, we can do things like we can log, obviously and you can see
6:20 that we can specify the level and this is the most generic way to write a log message,
6:24 but I wouldn't really use that most of the time,
6:28 what instead I would do is I would come down here and I would say either notice,
6:31 like if I want to set a notice level or maybe there is an error,
6:35 or there is an informational thing, or there is something critically bad
6:39 things like that, so instead of saying just "log",
6:43 which seems like a decent choice, I am going to say this is a notice,
6:47 so I like to have informational levels for stuff that I could really easily turn off and notice,
6:52 This is like just below warning because there is nothing wrong,
6:57 I just want to brace the startup of my app.
7:00 So, we are going to do it like this, alright, now let's go back
7:03 and because I changed this part right here, this needs to be a GET,
7:09 and instead of doing this "print 'Logging setup:'" like that here we can go ahead
7:17 and just let our log service handle that. In fact,
7:22 I think we could probably simplify this here as well,
7:26 I kind of like to be inexplicit, but let's go over here and in the rest of these places
7:32 we're doing prints, we could do something like this, so here we have a print,
7:36 we could say get our log service there and say log. this is a warning,
7:43 so we can warn and we don't need that warning anymore,
7:46 because warning is going to be actually inserted into the message by Logbook,
7:50 OK, that's cool, we are also printing down here,
7:54 let's go down and change this to say log.notice, right, there is nothing,
8:03 let me go up here yeah, that's a warning up there, great;
8:07 this is a notice again, this will be a warning,
8:13 take that out, alright, so we are going to warn if MailChimp is not set up
8:17 and finally, same thing for credit cards.
8:24 OK, it looks like we have replaced our print messages here,
8:28 let's go and run, and actually, let me change this as well, to not have a file name,
8:34 so we'll say there is None here, which or we could just cut it out,
8:40 and then in production, let me put this over in production as well,
8:47 in production, we're going to have this setup,
8:50 and maybe this is going to be NOTICE level,
8:55 and we want to go there, but here we are going to do INFO but have nothing, OK.
9:00 Let's give it a shot. Huh, remember, that other file I had,
9:07 I got to change it as well, but that should give us a chance to see the file logging in action,
9:12 so here check this out, I accidentally ran with the other not shown .ini file,
9:18 and you can see that we got our logging initialize, level is INFO,
9:24 so notices will show up, mode is hey, file mode and it went to that file.
9:30 Local, like in your web app folder, probably not the best place to put logs,
9:34 but I'm sure it's a bad place, but for now, it's working.
9:38 So we'll say this is running in production mode, excellent.
9:42 Now, I've changed that, let me run it again.
9:46 Maybe it didn't like my # None there, let's just put blank.
9:53 Here we go, so there is no file name,
9:56 so now we are in standard out mode, and app is running.
10:00 So that's the basic way of using Logbook, we come over here and let's just review,
10:06 we come over here, we set up the global stream handler
10:13 or basically the message handler either a stream handler
10:16 or a time rotating file handler or other things.
10:20 And then you just get a hold of log, by saying logger.... logbook.logger
10:26 allocate one of these, and notice it says "App" here and we are doing a notice log there
10:31 and if we go back to our console, you can see we have a notice from the app.
10:37 Alright, so this lets you really easily reference what part of the application is working,
10:42 we'll go back and we'll put some of this into our controller structure,
10:45 and there is a really nice way to bring that throughout the application
10:50 for all the web requests in just one single location.