Getting Started with NLP and spaCy Transcripts
Chapter: Part 4: NLP with huggingface and LLMs
Lecture: Boosting spaCy-LLM performance
Login or
purchase this course
to watch this video and the rest of the course contents.
0:00
There's one setting that I glanced over in the previous video, and that is this save I/O setting.
0:08
That setting actually allows us to do something extra from the notebook over here.
0:12
Now just to reiterate, I've got an NLP object here that really behaves as if it's loaded
0:18
from a normal spaCy model, but because I'm using spaCy LLM and because I've configured
0:23
everything the right way, all the NER predictions are really handled by an LLM.
0:30
But the look and feel really is just like you would normally have with spaCy. I get a doc object out over here with entities attached, etc.
0:39
But this doc object over here also has some extra data attached. So because we've configured it, we can have a look at the user data that's attached to
0:50
that document. And there is this LLM I/O key, and then this LLM key, and then a prompt key.
0:57
And if you inspect it, you'll notice there is a prompt being listed here.
1:04
The text that goes into the LLM over here is injected below there, but we definitely have a command that goes to the LLM.
1:13
Fun fact, by the way, believe it or not, adding this sentence saying that the LLM is an expert
1:20
named entity recognition system actually makes it a bit more performant.
1:25
It's one of these weird little trade tricks when you're designing prompts that seems to work.
1:30
After that, we again ask the LLM to give the response back in a certain structure, and we can actually also get the response from the LLM back as well.
1:42
The main difference is that we have to query the response key instead of the prompt key,
1:45
but also here we can see what the LLM is generating, which is something very similar to what we saw in ChatGPT.
1:54
So that's all pretty interesting, but you might wonder, well, is there maybe more information that we can give to the prompt?
2:03
Because right now the LLM, you could argue, doesn't have a whole lot of context to go by.
2:08
I am telling the LLM that there is a dish, but maybe I should also explain what I mean by this label dish. And the same thing with ingredients.
2:20
Is salt an ingredient? Is a sandwich a dish? There are these extra things that I might want to give the LLM in order for it to make
2:28
a better guess on what the entities are over here. As luck would have it, what I could also do is make a somewhat more advanced config file
2:37
where I pass that information along. So to help emphasize the comparison, this is the config file that we started with.
2:44
I'm using spaCy NER version 2 and, you know, just a few settings. But I can also add more settings here. For example, I can add label definitions.
2:57
So I could say, well, this is the name of a label, but here is just a little bit of
3:00
a description that hopefully will be able to push the LLM in the right direction.
3:06
I can add stuff like saying herbs and spices would make for good ingredients. What I'm even able to do is add a few examples.
3:18
So I've got this YAML file over here where I have a text example and I've got entities that it should detect.
3:27
The thinking here is that if you spot that the LLM makes a couple of mistakes, then you can add those mistakes here as an example.
3:35
And by adding that to the prompt, the hope is that the LLM won't make the same mistake again. So let's look at the effect of this.
3:44
I'm keeping basically everything the same, but I am going to point to the advanced configuration file instead. And here are the results.
3:54
The same entities are detected as before. So far, so good. But the one thing that's going to be very different now is this prompt.
4:04
Because we now have label descriptions, you can see that this part of the prompt is now being added.
4:10
We are telling the LLM that each label has a description that again have been written by an expert. Then another part of the prompt is added.
4:22
Here we can see that there are some examples to be used as a guide. At the end over here, we can see again that the response is exactly the same.
4:31
But the main thing, at least here, is that you do see that we're able to configure a more elaborate prompt.
4:38
In general, more elaborate prompts are more expensive because you are using more tokens, but they are also more accurate.
4:46
So getting that right will require a little bit of experimenting.
4:50
But the main thing that I do think is just kind of neat about this approach is that you
4:53
don't have to go and read the API documentation of all of these different LLM providers.
4:59
You have one consistent config file and this can communicate to a whole bunch of places.
5:05
If you're interested to learn more, if you go to the usage section of the docs, there
5:08
is this large language model section over here that you can click.
5:14
And besides giving a lot of extra context, you can also scroll down to this models bit over here at the end.
5:21
And here you can see this giant list of all the different models and backends that you can configure and use.
5:29
A lot of these are based on OpenAI models, but there's also some competitors like Cohere
5:33
and Anthropic, as well as some open source models like Dolly and Mistral.
5:40
As time moves forward, I'm sure more models will be added over here, but I do hope at least for now that you appreciate the separation of concerns.
5:48
You are able to pick from different providers and still get an LLM that integrates nicely with spaCy.