Build An Audio AI App Transcripts
Chapter: Feature 3: Summarize
Lecture: Calling the LeMUR API
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
All right, that was a lot of buildup to get everything ready to go, but we're now finally ready to use the magic of AI.
0:08
Up at the top, I've added import assemblyai.lemur, and let's also get lemur task response. We're gonna need those two things here in a second.
0:21
So down here, first we're gonna create a client. So I'll say lemur client, let's call it that to be super explicit. Say assemblyai.lemur.lemur class,
0:32
and now we just call a function on it. It will say response equals lemur client.task, and this is gonna be a lemur task response,
0:41
and just to make it really clear what this is so that we get autocomplete when we click on it, right? See, we get a response text and a response ID,
0:50
couple of extra things that come back from it, all right? But not a whole lot. Basically, we get the response as text.
0:59
All right, now what goes in here? I don't need that anymore. What goes in here? The prompt, and remember, we have two prompts.
1:07
Let's do the tldr prompt first. I'll say the final model. Now notice this is a lemur model. And notice this is a string enumeration
1:20
as we've seen before. So this is a, we have a basic model. We also have the cloud. We have a Mistral 7b and default. The one that I was actually using
1:29
for that really cool example I showed you was basic, so let's just keep going with that. You can play with this.
1:34
You can say the model output size is 2000, or the max output size is 2000 for a limit here. If you work with LLMs,
1:43
you may be familiar with temperature. Temperature is basically an expression of how creative do you want this to be
1:49
based on how much do you wanna just try to go straight down the narrow path of what you've asked it to do.
1:55
I'm gonna give it a little bit of leeway, but not too much. So we'll say 0.25. The final thing we want is input text
2:01
is gonna be this transcript that we've created. So notice right here, these are three additional ways that you can play with
2:09
what lemur is doing to see what kind of results you can get. You can try more creativity. Is it making up stuff? Maybe less creativity.
2:18
Do you wanna limit how much it can return to make it more concentrated? I think we're already doing that pretty well
2:24
with the last bit of our prompt here, but nonetheless, you can change this as well. And then also you can play with the model.
2:31
There's at least three options there. Pretty cool. This is what we're gonna do. Now, let's go ahead and run this and see what the response is
2:40
and when it comes back, what we're gonna do is I'm just gonna go to the database transcript. And on here, we have a TLDR summary,
2:49
and let's just say the response.response goes on there. We're not quite finished. There's one more thing we're gonna have to do
2:55
to clean up the response, but it'll be obvious in a minute. And let's just redo this again with the different prompt and the bullets
3:07
and the moments prompt. Okay, tighten that up a little just so it doesn't show us errors. And then when this is done, we've done both of these,
3:17
we'll just await saving that back to the database. Right? Active record style there. Couple of comments. Let's first put some print statements here.
3:29
Of course, this would be logging in a real app, but trying to keep it somewhat simple. So we'll say starting, say summarizing with Lemur TLDR mode,
3:40
and then let's go ahead and also say this here. Key moments. Just so when it's running, we can see what is happening here.
3:50
And we also have our timings. We'll see how long it took. It says transcript, but we'll put some right here.
3:56
All right, now it's time for the big moment. Let's go and run it. Let's go and see if we can learn more about this Emma Stone episode.
4:06
Sure enough, there it is. Refresh just to see everything's good. So we already have the transcript, so that should be set,
4:14
but we don't have the summary. And let's click that and see what we get. Starting. Summarizing Lemur TLDR mode. Key moments mode.
4:35
This is real time. I'm not speeding anything up. I'll just let it go at real time so you can see how long this takes.
4:41
23 seconds to do both of those things, which is excellent. And you see this is almost an hour's worth of text. So that's pretty excellent.
4:48
Now, if we refresh, the website actually already knows how to render this. As you've seen it a few places when it was already in the database.
4:56
Check out what we got. First thing to notice, here's a five sentence summary of some number as in the form of it. We don't wanna show that to the user.
5:05
So let's just go ahead and refresh. As in the form of it, we don't wanna show that to the user. That's some sort of leakage. The same thing.
5:12
Here's a bullet point. So let's go from here and we'll deal with that first sentence of both sections later.
5:18
Emma Stone discusses her Oscar nominated role in the film, Poor Things, where she plays a woman brought back to life
5:25
by a mad scientist with an infant's brain. She talks about her character's journey, developing adult cognition and sexuality, et cetera, et cetera.
5:34
Let's see about here. Nominated for Poor Things. In her film, her character, Bella, struggles to understand these things. Pretty interesting.
5:46
Stone and the host discuss some of the provocative representations of female sexuality and independence. Listeners get a fascinating look
5:54
at Stone's breakthrough role in the film and her ongoing journey with mental health issues. Excellent, right? Look at this. This is really cool.
6:02
Now, how much better is that than Emma Stone is nominated for Poor Things? She spoke to Terry Gross about her relationship with anxiety
6:09
and somebody reviews the thing we probably don't care about. Also, this person talks about his book. I really like how this came out.
6:17
Now, this bit and this bit, we're gonna want to come back and deal with, but yeah, this works really, really well.
6:25
It took about 11 seconds because it was 23 for both and they're about equi-time. So let's look at one real quick thing before we wrap up this part.
6:35
You may have noticed this comment. By the way, lemur.task is blocking. Hmm, that doesn't sound good.
6:41
That 11 seconds wasn't that long, but you know what? FastAPI could not do any more. Like, if we go back here, you'll notice something
6:49
in the search engine is kind of cluttering up our history, but notice this. This is 23 seconds and this polling,
7:00
is it done yet, is it done yet, is it done yet? That happens every two or three seconds. Remember on the transcript, it goes,
7:08
is it done, is it done, is it done? Here, no, it couldn't do that. Why? Because this blocking call is blocking up the entire async IO event loop.
7:18
Mm-hmm, that's not our deal. Our website has lost a tremendous amount of scalability when it can't process any other requests
7:27
for basically 11 seconds at a time. So we should call the async version, yes? And that's what we did with the transcript.
7:35
And we used our run future at the bottom, just right down there, and we awaited that. That was cool. The SDK does not have anything other
7:44
than that synchronous version, okay? There will be, I've been working with the Assembly AI folks and said, hey, you know,
7:51
you really should have an async version because this is important. And they're gonna add a task async, most likely,
7:58
not 100% guarantee, but quite, quite likely there will be a task async, in which case, when there is a task async, just like this,
8:07
you can create, get the future back, and you can await run future, and that'll keep FastAPI and anything doing async IO cruising right along,
8:16
'cause it will be running somewhere else outside of this, and you're just awaiting its completion. I'll just put, I can't leave it this way
8:24
'cause it's not working. I'll come back and fix this code. I will change this code if they add an async version. Here we go.
8:31
So I'll put a little example here. Please use future equals lemur-client.task.async, response equals await run future of that name,
8:40
and however you like, but there should be some pattern like this that you can use eventually. There's other things we could do to run this
8:47
in a non-blocking way, but hopefully the folks over at Assembly AI are able to get that into their SDK
8:52
and make things work just a little bit better for us. Here we go though. Besides that little thing I wanna put on your radar
9:00
to make sure you're aware of what's going on there, 'cause it's not entirely obvious. That's it. Come up with our prompt. We send it over here.
9:07
Perfect.