Ensuring Independent API Calls to Ollama

At a glance

The post asks if there is a way to ensure no shared context is maintained between calls to the Ollama API, so that each call is treated as independent. The comments discuss different approaches to this:

One community member suggests using a chat engine that can maintain context, but the original poster clarifies that they do not want to maintain any context at all. Another community member suggests that updating the message block with each new query may help treat each call as independent.

There is no explicitly marked answer, and the community members seem to have differing opinions on whether maintaining some context can be beneficial for the responses.

Useful resources

mmostlyAdi

Is there a way to ensure no shared context is maintained between calls, and every api call to ollama is treated as independent ?

7 comments

WWhiteFang_Jr

You can use chat engine: https://docs.llamaindex.ai/en/stable/module_guides/deploying/chat_engines/

This gives you the feature to maintain the context

mmostlyAdi

I do not want to maintain context at all, every call should be unique
Right now I use something like this
from ollama import Client

Client.chat(
model="llama3.2:3b",
messages=[
{"role": "user", "content": question},
],
optiions={
"frequency_penalty": 0,
"stop_sequences": ["Thank you", "Best regards"],
},
)

WWhiteFang_Jr

oh damn, I missed the 'no' 😆

WWhiteFang_Jr

I think if you keep updating the message block once you recieve the response ( i,e remove previous message and then send only the new query )

Ollama would consider that as a independent request only

mmostlyAdi

Even I agree, but In my answer. I feel the responses get improves when I have a continous chat.

mmostlyAdi

Have you found something in documentaio which show this ?

WWhiteFang_Jr

This look right to me tbh 👀

Add a reply

Find answers from the community

Ensuring Independent API Calls to Ollama