LlamaIndex
Log in

Find answers from the community

Updated 3 months ago

Hello Logan M

TThomas1234
·
Hello
When using the query engine with streaming set to True on your index how can you avoid having the </S> at the end of your answer ?
Thanks for the help
L
T
5 comments
LLogan M
are you using a custom LLM?
TThomas1234
Yes I'm using a finetuned version of Llama 2
LLogan M
hmm, likely your best bet is just removing it with python string parsing haha
TThomas1234
Okay thanks ! I was wondering if there a command to remove special tokens 😅
LLogan M
HuggingFace has a few utils for it with their tokenizer, but not sure what your setup looks like
Add a reply
Sign up and join the conversation on Discord