Build An Audio AI App Transcripts
Chapter: Feature 3: Summarize
Lecture: Cleaning up LLM Prompt Leakage
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
Well, we have our data being generated by the Lemur LLM, and it's being added here just like we hoped in our summary.
0:10
But we did see that there's this confirmation statement at the beginning of both AI responses. That's not gonna come across great for our product.
0:20
I mean, people know it's an AI summary, that's kind of the point, but you don't wanna leak the prompt engineering back. So what to do about this?
0:30
Well, what we're gonna do is we're going to go and find a way to replace this. And because this could vary, right?
0:37
It could say, here's a seven sentence. I said the response could be five to eight sentences. Maybe it'll say it's eight, who knows?
0:45
And this could vary as the LLM changes over time. So I wanna use a regular expression. Regular expressions can be super tricky.
0:52
If you're not familiar with them, honestly, you can go to chat.gtp or one of the local models like LM Studio and just say, I have this text,
1:01
I need to match it with a regular expression. They're actually really good at that. So what we're gonna do is we're gonna just remove
1:07
that section and we're gonna remove that as well there. All right, here we are in that code that we just wrote. That's not the right one.
1:17
Here we are, here we are in the code we just wrote, worker summarize episode. Remember down here at the end, we have this section
1:27
and maybe I'll make it its own little separate thing. What number are we on just to be consistent? Six, step six.
1:34
So I put these examples here like this. Now I'll just create the regular expression here and then maybe move it up so we can compile it
1:43
and just use it once. So we're gonna import RE, which I'll put at the top in a second. And we'll say RE compile.
1:50
And here's where things get interesting. So what I'm gonna do is I'm gonna say, here's a some sort of sentence. (mouse clicking) Like this.
2:00
And we'll have onto the end and down to just say colon, matching the first colon. So what we can say is we can say that the text starts with that.
2:17
Says here's a, now this number could change. We can say it's any digit. And the way that you do that is you say bracket zero dash nine
2:26
so any characters between zero and nine. And there could be potentially more of them if it's a 10, which we're gonna see in just a second below.
2:35
And then we wanna space sentence and then we can have arbitrary text. So the way we say that is dot would be one, plus would be one or more, onward.
2:46
All right, so we'll call this reg X equals that. And then let's just do a replacement here like that. We'll say, okay, this thing is actually reg X.
2:57
Look at that. PyCharm is saying, you know what? You probably wanna substitute that, don't you? Reg X dot sub. And in fact, that's very, very close.
3:05
So I'm gonna let it write that. And what we want here is the summary bullets like this. Incredible actually. So the API here is a little bit funky.
3:15
We're gonna take this pattern, and that's regular expression. We want to substitute, and we wanna use this new thing into that old thing.
3:24
I kind of feel like this is backwards, but anyway, that's fine. Summary, TLDR, and summary, TLDR actually. We'll do that. And let's do another here.
3:42
This is going to be, here's a 10 bullet point. Bullet point, summary, right? That should work. Of course, we could make this more general,
3:58
but I kind of want it to be really restrictive and not possibly hit the wrong thing. Let's see how this works,
4:03
and then I'll move that code a little bit around. I wanted to keep it kind of local while we were looking at it.
4:08
And in fact, so we don't have to keep running over and over. Let's do one more thing with this. Let's move this.
4:14
Let's move this code over into the view model that shows that. So that's the episode view. So we're gonna go over here to the view models
4:23
for the podcast episode. And let's just say if self.ai summary, this way we can just do it on the response here.
4:32
And the way this thing works, it stores both of these. So let's see, not, tldr, and this one will be bullets.
4:49
All right, so that'll just do it on the already generated text. Let's see how we did. Oh, it looks like that didn't take.
5:03
Now, I think it's just that I'm being overly restrictive and saying it has to start with this. Let's try that real quick. And sure enough, there we go.
5:16
Look, that chunk is gone. That restatement is gone. I imagine it probably is. There's like some kind of spacer or something here. We could try .strip.
5:28
Just, I wanted to kind of see if that regex would have been just fine. Let's see. Yeah, sure. See, yeah, sure enough.
5:40
There was like some kind of space there at the beginning and I suppose it doesn't really hurt to do a strip on these as well.
5:48
Okay, so it looks like that regular expression is working and I'm gonna take this away. Just wanted to make sure that, you know,
6:04
we don't have to go wait a minute at a time to do this test. All right. So we'll go back to the generation section here
6:11
and this, I'll just do strip on each of these. In fact, let's do it where we originally set it. It's all about the data cleaning. Okay.
6:24
Yeah, these look good. Let's go ahead and move this up here and we'll call this regex_tldr. And this will be regex_moments.
6:38
And let's put this up at the top. Here we go. So why am I putting this up at the top here? Well, because when you compile a regular expression,
6:53
this takes a little bit of memory, takes a little bit of time. And honestly, we don't need to do this each time. The regular expression doesn't change.
7:00
So, you know, tldr, not misspelled. Everything is fine here. So let's just go and take it through its paces completely one time here.
7:09
Let's go summarize an episode. How about Inside Azure Data Centers with Mark Russinovich? We'll go and first create the transcript when we click this,
7:24
and then we'll create the summary. And hopefully everything looks right without those leaky LLM restatements there. So hang tight.
7:31
I'll zoom this ahead. When it's done, we'll have a look. Well, here it is. It's all finished. Let's go and see how it came out. Perfect.
7:41
Well, it's almost perfect. It didn't get the guest, the host right now, did it? I mean, maybe Peter Bright's awesome,
7:52
and I should be flattered that it put me and replaced me with Peter Bright, but I have no idea. So anyway, let's see what we got here.
8:00
In this episode of ""Talk Python to Me, the host discusses his interview with Microsoft Azure CTO Mark Russinovich from inside an Azure data center.
8:12
Now this is not good. This is not, that is not great, but let's look at the moments here. Microsoft Azure CTO Mark Russinovich
8:23
joins the podcast to give a rare look inside Azure's massive global data center operations. Yep, absolutely, that's good.
8:30
Azure originated in 2006 as a small incubating team within Microsoft and launched publicly in 2008 before its full commercial launch in 2010
8:42
when Russinovich joined. Also, I believe that's correct. In its early years, Azure ran on only a few thousand servers
8:49
across two regions, but has since exploded with millions of servers, now spanning 60 worldwide region data centers. Yep.
8:58
A new AI chip called Azure Cobalt is optimized to accelerate machine learning, now leverages Python and AI Assistant Co-Pilot
9:08
for his own research on the massive Azure platform. The key moments is perfect. TLDR, I'm not so sure about this. So let's try another one.
9:16
We'll see what reporter I get substituted as. There, let's look at a few more here. How about, how about celebrating JupyterLab? Let's do that.
9:26
Hey, it's done. Let's see how we did. First of all, will the restatements be gone? Yes, they are. Look at that. So what is the summary?
9:38
On a recent Talk Python to Me podcast, developers of the Jupyter Project, now there were three or four folks who were on the Jupyter Project,
9:46
and we came and we talked about JupyterLab 4 being released as well as the Python team. So we're gonna talk about that.
9:52
So we're gonna talk about the Python team and the Jupyter Project, and we came and we talked about JupyterLab 4 being released as well.
9:59
Jupyter 7, I believe, and a couple of other things. So let's see. JupyterLab 4.1 introduces features like code history restoration
10:08
and markdown header collapsing. Upcoming iterations with AI tools aimed to autocomplete code and provide hints to the Python coders.
10:18
All right, that looks pretty good. What about the moments? Jeremy, Sylvan, and Frederick, I believe that is exactly right,
10:24
and the versions are right, perfect. Darian worked on Jupyter for this time. At JupyterCon, Darian and Martha introduced JupyterLab 4.
10:32
That's true, and very, very neat. And we talked a lot about their presentation there, and then some of the things they discussed.
10:41
See, real-time collaboration is being tackled. The tools like Google Docs show the promise for co-editing code.
10:50
One of the examples we discussed in the episode was that JupyterLab is really just a bunch of plugins or extensions made on top of Jupyter
10:58
to make it what it is, and said, Hey, you know, another example of something you could do with that is this thing called JupyterCAD for 3D modeling.
11:04
Yeah, that's true. JupyterLite and desktop apps. Yeah, we talked about how you could use JupyterLite and not have to have Python or install anything
11:11
or run a server. Yeah, I think this is solid right here. I feel like there might be something that was a chunk of the conversation
11:21
that didn't get captured, but we're trying to have it limit that down, right? So yeah, I'm happy with this. It's better than what I put, right?
11:29
Here you can see the summary, the show notes that I gave it, and the links from the show. So yeah, we got a lot more information about this right here.
11:39
Excellent. But the most important part, not just is this right, we already saw this before. What we saw that is the LLM restatement of like,
11:51
Hey, I understood you. Now here's your answer. That part is gone because we were able to do our regular expressions right here.
11:58
Here is a some number of sentence, or here are, here's a some number, bullet points, summary, et cetera. All right, well, there it is.
12:08
I think we have this feature all done.