Find answers from the community

Updated 11 months ago

does anybody knows how to set up the llm

At a glance

The community member is asking how to set up the LLM parameters (temperature, top_p, and max output tokens) when using the Ollama model. The code provided shows how to load the Ollama model with a specific model name and request timeout. A community member responds that the temperature and other values can be passed as keyword arguments, providing an example of how to set the temperature to 0.8.

Useful resources

gghxsted.

does anybody knows how to set up the llm parameters (temperature, top_p and max output tokens) when using ollama to load a model ?
this is the code :

Plain Text

from llama_index.llms.ollama import Ollama

llm = Ollama(model="llama2", request_timeout=60.0)

response = llm.complete("")
print(response)

1 comment

WWhiteFang_Jr

You can pass temperature and other values as kwarg,
like this:

Plain Text

llm = Ollama(model="llama2", request_timeout=60.0, temperature=0.8)

https://github.com/run-llama/llama_index/blob/2b77f89775840d6b796bcc693f7593d2aebc5fec/llama-index-integrations/llms/llama-index-llms-ollama/llama_index/llms/ollama/base.py#L56

Add a reply