Find answers from the community

Updated 8 months ago

Here is another questions. Ollama has

At a glance

The community member is asking about the use of the "OLLAMA_NUM_PARALLEL" environment variable in Llama-Index, which was recently introduced by Ollama to allow multiple models to run concurrently. The community member has not seen explicit support for this feature in Llama-Index and is wondering if there are any experimental features or branches attempting to incorporate it.

In the comments, another community member mentions that they have not tried it yet, and another suggests that nothing needs to change in Llama-Index to use this feature, as it is just an environment variable. The third comment indicates that the feature is experimental and that Ollama may want to automatically set the variable based on hardware data in the future.

There is no explicitly marked answer in the comments.

Here is another questions. Ollama has recently introduced "OLLAMA_NUM_PARALLEL" to allow multiple models to run concurrently. However, I have not seen explicit support for this in Llama-Index. Do you know if there are any experimental features or branches attempting to incorporate the use of OLLAMA_NUM_PARALLEL? Thanks.
L
3 comments
First i've heard about it, so haven't tried it yet.
It sounds like nothing needs to change in llama-index to use this, its just an env variable?
(experimental as well, sounds like they want to automatically set this in the future based on hardware data)
Add a reply
Sign up and join the conversation on Discord