LlamaIndex
Log in

Find answers from the community

Updated 9 months ago

Keyword

NNiels
·
Is it a known issue that when using the keyword extractor for document with some english terms but mostly other content it stores all of the keywords in english?

Or maybe if we wanted to fix that we would need to make a custom extractor?
L
N
10 comments
LLogan M
That makes sense to me since the prompt is in English.

You'd probably have to rewrite the extraction prompt in another language?
NNiels
Yeah will probably have to add "make sure to keep the keywords in the original language" or something
NNiels
Probably doesn't make sense to add to llama index itself?
LLogan M
Oh lol the template is hard-coded
https://github.com/run-llama/llama_index/blob/78af3400ad485e15862c06f0c4972dc3067f880c/llama-index-core/llama_index/core/extractors/metadata_extractors.py#L195
NNiels
Yeah i noticed
LLogan M
Yea maybe. In general multilingual isn't really accounted for anywhere
NNiels
Kind of a limitation to be fair
NNiels
But i guess a custom keyword extractor would resolve that for us
LLogan M
Yea I agree -- in general, it's on the roadmap to centralize all prompts into some central registry to make things easier to customize.
NNiels
Cool!
Add a reply
Sign up and join the conversation on Discord