Find answers from the community

Updated 6 months ago

Here is another questions. Ollama has

Here is another questions. Ollama has recently introduced "OLLAMA_NUM_PARALLEL" to allow multiple models to run concurrently. However, I have not seen explicit support for this in Llama-Index. Do you know if there are any experimental features or branches attempting to incorporate the use of OLLAMA_NUM_PARALLEL? Thanks.
L
3 comments
First i've heard about it, so haven't tried it yet.
It sounds like nothing needs to change in llama-index to use this, its just an env variable?
(experimental as well, sounds like they want to automatically set this in the future based on hardware data)
Add a reply
Sign up and join the conversation on Discord