Python for Entrepreneurs Transcripts
Chapter: Monitoring and logging in production
Lecture: Integrating logbook into our app
Login or
purchase this course
to watch this video and the rest of the course contents.
0:02
We've set up the structure of where we are going to store our information, read it in from the config file and whatnot, let's use Logbook now.
0:09
So, this is probably no surprise, we'll start by saying "import logbook". Now, remember, this needs to be put into our setup requirements,
0:19
this needs to be literally installed because we don't have it yet, so first, let's install the package, thank you PyCharm, ta-da, OK,
0:29
I got to move away for a second, and then come up here, hit it again and I can add it to the setup, and then it's saying we're not using it
0:37
but that's cool because we are going to that now. So the first thing that we want to think about is where do we want the logs to go,
0:44
do we want them just to go to the console, do we want them to be sent to a file? Things like that,
0:52
and my perspective here is in production, we want them to go to a file and in development we just want them to go to the console,
0:59
so we'll set that up now. Now this level when it comes in here is going to be a text string straight out of the config file,
1:09
Logbook has its own enumeration, which represents this, so I am going to make this very clear and say log_level_text,
1:16
we are going to need a function to convert that to Logbook levels, so let me go and just paste that function in because once you see it, you'll be like
1:24
"wow", there is not point watch me write that, so here what we are going to do is we are just going to have a function,
1:30
pass it a string, and it's going to come in and it's going to pull out logbook.CRITICAL,
1:34
if it sees the string CRITICAL, Logbook.ERROR, if it sees the string ERROR and so on. And we are also stripping it and uppercasing it,
1:42
so there is a little bit of non equalization going on. OK, so the first thing we are going to do is say "level...",
1:51
so we want to come here and say get the Logbook level and we are going to pass in that text and it's going to convert it to that enumeration.
2:00
The next thing we got to do is decide where do we go, where do we pass this information,
2:08
we could do a couple of things, we could queue this on the filename, or we could also pass the development mode or whatever,
2:14
I am going to basically say if you supply filename, it's going into the file, if you don't supply filename, it's going to go to the console
2:20
so we'll say something like this, "if filename", now let's start with "not", so standard logging, right,
2:26
so if it doesn't go to the filename, we are going to say Logbook and we are going to setup a stream handler,
2:33
and the stream we want to go to is standard out, which we can get from sys, we'll say standard out, so that's just console,
2:40
and we want to set the level to level, so remember, if I log in informational message,
2:48
but I say I only want to see streamed to the standard out, stuff that's warnings and errors,
2:55
that message will basically silently go away, which is really nice because that lets us turn up and down the logging just in one place
3:01
and the rest of the app doesn't have to worry about it, right, they log at whatever levels they think make sense, and then here we just adjust it.
3:08
And so we can set this up and then we need to push this to the application, so basically that means make this globally available
3:16
as a stream handler for the rest of the application. So later anywhere in our app we can just create a Logbook,
3:24
set some values and start going, and it will use this stream handler. So we'll come down here and say if it's not the case we have no file,
3:32
we do actually want to go to the file system. Now, we could just go straight and log to a file, but on busy application
3:41
that turns out to be a huge problem, like on Talk Python To Me, recently I went and checked my logs, I had something like 3 GB of log text,
3:51
that's a serious amount of logging, right, and you want to clean that out every now and then,
3:57
if this is one huge 3 GB text file, that's going to be a problem, trust me. So what we are going to do is we are going to set this not just to a file
4:04
but we set to a timed rotation file handler, so basically we get a log file every day, the base name plus the date.
4:14
Again, we are going to set the level, and then we are going to set a date format, let me format it like this, so we are going to print it out,
4:30
store the date in year-month-day format and again, in this case we want to push this to the application,
4:37
so nowhere outside of this little block of code, do we care or even know about how the logs are written; we'll say here if you don't provide us a file,
4:50
we're just going to stream it to the standard out, but if you do provide us a file,
4:53
we are going to create a rotating file based on that base filename that you provided us. Alright, so that's all good.
5:01
Now that this is in place, we can go over here and write this little startup log, so when we go and try to access to the startup log,
5:08
we are going to return an instance of a Logbook, and that is going to be of course leveraging this
5:13
but we don't have to directly say anything, so we are going to create a new Logbook,
5:17
and the thing that you actually create is a logger and in here you give it a name
5:22
and so this is like the area of your application, what part of your app is doing this,
5:28
so this remember is going to be used sort of through the startup process and nowhere else so I am going to call this,
5:34
I'm just going to call it "App", right, this is the application startup level.
5:38
Let's go back up here one more time, and let's use our little startup log
5:41
that we are creating, and just print a message to say "hey, everything is working",
5:47
so we'll say the message is..., so we'll say logging initialize with the level and the mode, whether or not it's in file mode or standard out mode.
6:00
OK, and then, we're going to say LogService, let's make that static, actually let me change that to a static method.
6:16
Now, we can do things like we can log, obviously and you can see that we can specify the level and this is the most generic way to write a log message,
6:25
but I wouldn't really use that most of the time, what instead I would do is I would come down here and I would say either notice,
6:32
like if I want to set a notice level or maybe there is an error, or there is an informational thing, or there is something critically bad
6:40
things like that, so instead of saying just "log", which seems like a decent choice, I am going to say this is a notice,
6:48
so I like to have informational levels for stuff that I could really easily turn off and notice,
6:53
This is like just below warning because there is nothing wrong, I just want to brace the startup of my app.
7:01
So, we are going to do it like this, alright, now let's go back and because I changed this part right here, this needs to be a GET,
7:10
and instead of doing this "print 'Logging setup:'" like that here we can go ahead and just let our log service handle that. In fact,
7:23
I think we could probably simplify this here as well, I kind of like to be inexplicit, but let's go over here and in the rest of these places
7:33
we're doing prints, we could do something like this, so here we have a print, we could say get our log service there and say log. this is a warning,
7:44
so we can warn and we don't need that warning anymore, because warning is going to be actually inserted into the message by Logbook,
7:51
OK, that's cool, we are also printing down here, let's go down and change this to say log.notice, right, there is nothing,
8:04
let me go up here yeah, that's a warning up there, great; this is a notice again, this will be a warning,
8:14
take that out, alright, so we are going to warn if MailChimp is not set up and finally, same thing for credit cards.
8:25
OK, it looks like we have replaced our print messages here, let's go and run, and actually, let me change this as well, to not have a file name,
8:35
so we'll say there is None here, which or we could just cut it out, and then in production, let me put this over in production as well,
8:48
in production, we're going to have this setup, and maybe this is going to be NOTICE level,
8:56
and we want to go there, but here we are going to do INFO but have nothing, OK. Let's give it a shot. Huh, remember, that other file I had,
9:08
I got to change it as well, but that should give us a chance to see the file logging in action,
9:13
so here check this out, I accidentally ran with the other not shown .ini file, and you can see that we got our logging initialize, level is INFO,
9:25
so notices will show up, mode is hey, file mode and it went to that file. Local, like in your web app folder, probably not the best place to put logs,
9:35
but I'm sure it's a bad place, but for now, it's working. So we'll say this is running in production mode, excellent.
9:43
Now, I've changed that, let me run it again. Maybe it didn't like my # None there, let's just put blank. Here we go, so there is no file name,
9:57
so now we are in standard out mode, and app is running. So that's the basic way of using Logbook, we come over here and let's just review,
10:07
we come over here, we set up the global stream handler or basically the message handler either a stream handler
10:17
or a time rotating file handler or other things. And then you just get a hold of log, by saying logger.... logbook.logger
10:27
allocate one of these, and notice it says "App" here and we are doing a notice log there
10:32
and if we go back to our console, you can see we have a notice from the app.
10:38
Alright, so this lets you really easily reference what part of the application is working,
10:43
we'll go back and we'll put some of this into our controller structure, and there is a really nice way to bring that throughout the application
10:51
for all the web requests in just one single location.