Find answers from the community

Updated 4 months ago

Are there some methods for speeding up

At a glance

The community member is asking for methods to speed up the "synthesize" process, noting that the LLM (Large Language Model) calls within "synthesize" are the most time-consuming part. In the comments, another community member suggests two ways to potentially speed it up: getting a faster LLM or decreasing the amount of data being sent to the LLM (by adjusting the chunk size or top-k parameters).

Are there some methods for speeding up synthesize? the llm calls within synthesize are the most timely in my rag when looking at tracing via phoenix
L
1 comment
Only way to speed it up is to get a faster LLM πŸ˜‰ Or decrease the amount of data being sent to the LLM (chunk size/top-k)
Add a reply
Sign up and join the conversation on Discord