Find answers from the community

x
xKwan
Offline, last seen 3 months ago
Joined September 25, 2024
x
xKwan
·

Vllm

Hello, does anyone know how to use prompt template with vllm?
https://docs.llamaindex.ai/en/stable/api_reference/llms/vllm/

It seems that messages_to_prompt and completion_to_prompt is added when loading up LLM, but you can't give few shot learning during runtime.
2 comments
x
L
I am trying to get the node score but it always show as 1, where score 0 means 0% relevant and 1 means 100% relevant.
11 comments
x
O
L