Build An Audio AI App Transcripts
Chapter: Feature 3: Summarize
Lecture: Calling the LeMUR API

Login or purchase this course to watch this video and the rest of the course contents.
0:00 All right, that was a lot of buildup to get everything ready to go, but we're now finally ready to use the magic of AI.
0:08 Up at the top, I've added import assemblyai.lemur, and let's also get lemur task response. We're gonna need those two things here in a second.
0:21 So down here, first we're gonna create a client. So I'll say lemur client, let's call it that to be super explicit. Say assemblyai.lemur.lemur class,
0:32 and now we just call a function on it. It will say response equals lemur client.task, and this is gonna be a lemur task response,
0:41 and just to make it really clear what this is so that we get autocomplete when we click on it, right? See, we get a response text and a response ID,
0:50 couple of extra things that come back from it, all right? But not a whole lot. Basically, we get the response as text.
0:59 All right, now what goes in here? I don't need that anymore. What goes in here? The prompt, and remember, we have two prompts.
1:07 Let's do the tldr prompt first. I'll say the final model. Now notice this is a lemur model. And notice this is a string enumeration
1:20 as we've seen before. So this is a, we have a basic model. We also have the cloud. We have a Mistral 7b and default. The one that I was actually using
1:29 for that really cool example I showed you was basic, so let's just keep going with that. You can play with this.
1:34 You can say the model output size is 2000, or the max output size is 2000 for a limit here. If you work with LLMs,
1:43 you may be familiar with temperature. Temperature is basically an expression of how creative do you want this to be
1:49 based on how much do you wanna just try to go straight down the narrow path of what you've asked it to do.
1:55 I'm gonna give it a little bit of leeway, but not too much. So we'll say 0.25. The final thing we want is input text
2:01 is gonna be this transcript that we've created. So notice right here, these are three additional ways that you can play with
2:09 what lemur is doing to see what kind of results you can get. You can try more creativity. Is it making up stuff? Maybe less creativity.
2:18 Do you wanna limit how much it can return to make it more concentrated? I think we're already doing that pretty well
2:24 with the last bit of our prompt here, but nonetheless, you can change this as well. And then also you can play with the model.
2:31 There's at least three options there. Pretty cool. This is what we're gonna do. Now, let's go ahead and run this and see what the response is
2:40 and when it comes back, what we're gonna do is I'm just gonna go to the database transcript. And on here, we have a TLDR summary,
2:49 and let's just say the response.response goes on there. We're not quite finished. There's one more thing we're gonna have to do
2:55 to clean up the response, but it'll be obvious in a minute. And let's just redo this again with the different prompt and the bullets
3:07 and the moments prompt. Okay, tighten that up a little just so it doesn't show us errors. And then when this is done, we've done both of these,
3:17 we'll just await saving that back to the database. Right? Active record style there. Couple of comments. Let's first put some print statements here.
3:29 Of course, this would be logging in a real app, but trying to keep it somewhat simple. So we'll say starting, say summarizing with Lemur TLDR mode,
3:40 and then let's go ahead and also say this here. Key moments. Just so when it's running, we can see what is happening here.
3:50 And we also have our timings. We'll see how long it took. It says transcript, but we'll put some right here.
3:56 All right, now it's time for the big moment. Let's go and run it. Let's go and see if we can learn more about this Emma Stone episode.
4:06 Sure enough, there it is. Refresh just to see everything's good. So we already have the transcript, so that should be set,
4:14 but we don't have the summary. And let's click that and see what we get. Starting. Summarizing Lemur TLDR mode. Key moments mode.
4:35 This is real time. I'm not speeding anything up. I'll just let it go at real time so you can see how long this takes.
4:41 23 seconds to do both of those things, which is excellent. And you see this is almost an hour's worth of text. So that's pretty excellent.
4:48 Now, if we refresh, the website actually already knows how to render this. As you've seen it a few places when it was already in the database.
4:56 Check out what we got. First thing to notice, here's a five sentence summary of some number as in the form of it. We don't wanna show that to the user.
5:05 So let's just go ahead and refresh. As in the form of it, we don't wanna show that to the user. That's some sort of leakage. The same thing.
5:12 Here's a bullet point. So let's go from here and we'll deal with that first sentence of both sections later.
5:18 Emma Stone discusses her Oscar nominated role in the film, Poor Things, where she plays a woman brought back to life
5:25 by a mad scientist with an infant's brain. She talks about her character's journey, developing adult cognition and sexuality, et cetera, et cetera.
5:34 Let's see about here. Nominated for Poor Things. In her film, her character, Bella, struggles to understand these things. Pretty interesting.
5:46 Stone and the host discuss some of the provocative representations of female sexuality and independence. Listeners get a fascinating look
5:54 at Stone's breakthrough role in the film and her ongoing journey with mental health issues. Excellent, right? Look at this. This is really cool.
6:02 Now, how much better is that than Emma Stone is nominated for Poor Things? She spoke to Terry Gross about her relationship with anxiety
6:09 and somebody reviews the thing we probably don't care about. Also, this person talks about his book. I really like how this came out.
6:17 Now, this bit and this bit, we're gonna want to come back and deal with, but yeah, this works really, really well.
6:25 It took about 11 seconds because it was 23 for both and they're about equi-time. So let's look at one real quick thing before we wrap up this part.
6:35 You may have noticed this comment. By the way, lemur.task is blocking. Hmm, that doesn't sound good.
6:41 That 11 seconds wasn't that long, but you know what? FastAPI could not do any more. Like, if we go back here, you'll notice something
6:49 in the search engine is kind of cluttering up our history, but notice this. This is 23 seconds and this polling,
7:00 is it done yet, is it done yet, is it done yet? That happens every two or three seconds. Remember on the transcript, it goes,
7:08 is it done, is it done, is it done? Here, no, it couldn't do that. Why? Because this blocking call is blocking up the entire asyncio event loop.
7:18 Mm-hmm, that's not our deal. Our website has lost a tremendous amount of scalability when it can't process any other requests
7:27 for basically 11 seconds at a time. So we should call the async version, yes? And that's what we did with the transcript.
7:35 And we used our run future at the bottom, just right down there, and we awaited that. That was cool. The SDK does not have anything other
7:44 than that synchronous version, okay? There will be, I've been working with the Assembly AI folks and said, hey, you know,
7:51 you really should have an async version because this is important. And they're gonna add a task async, most likely,
7:58 not 100% guarantee, but quite, quite likely there will be a task async, in which case, when there is a task async, just like this,
8:07 you can create, get the future back, and you can await run future, and that'll keep FastAPI and anything doing asyncio cruising right along,
8:16 'cause it will be running somewhere else outside of this, and you're just awaiting its completion. I'll just put, I can't leave it this way
8:24 'cause it's not working. I'll come back and fix this code. I will change this code if they add an async version. Here we go.
8:31 So I'll put a little example here. Please use future equals lemur-client.task.async, response equals await run future of that name,
8:40 and however you like, but there should be some pattern like this that you can use eventually. There's other things we could do to run this
8:47 in a non-blocking way, but hopefully the folks over at Assembly AI are able to get that into their SDK
8:52 and make things work just a little bit better for us. Here we go though. Besides that little thing I wanna put on your radar
9:00 to make sure you're aware of what's going on there, 'cause it's not entirely obvious. That's it. Come up with our prompt. We send it over here.
9:07 Perfect.


Talk Python's Mastodon Michael Kennedy's Mastodon