Find answers from the community

Updated 6 months ago

Speed

At a glance

The community member is asking if it is reasonable for a Llama chatbot to take 19 seconds on average to respond, and if not, what they should do to speed it up. Another community member responds that the response time can depend on the busyness of the OpenAI servers and the number of language model calls required for the query, which depends on the index/query setup. There is no explicitly marked answer in the comments.

hi, guys, is it reasonable for llama chatbot to take 19 secs on average to make response? if not, what should i do to speed it up?? thanks....
L
C
2 comments
A lot of it depends on 1. How busy openai servers are and 2. How many llm calls one query takes (which depends on your index/query setup)
@Logan M thank u so much👍
Add a reply
Sign up and join the conversation on Discord