Find answers from the community

Updated 6 months ago

hello @everyone i have doubt how to set

At a glance

The community member has a doubt about how to set vLLM server endpoints to Settings.llm. A comment suggests that the community member needs to create a vLLM instance and add it to Settings, with the example of Settings.llm = vLLM instance. The comment also provides a link to relevant documentation. There is no explicitly marked answer in the provided information.

Useful resources
hello @everyone i have doubt how to set vllm server endpoints to Settings.llm , please help
W
1 comment
You'll have to create vLLM instance and add it to Settings.

Settings.llm = vLLM instance here
https://docs.llamaindex.ai/en/stable/examples/llm/vllm/?h=vllm#completion-response
Add a reply
Sign up and join the conversation on Discord