LlamaIndex
Log in

Find answers from the community

Updated 2 months ago

Hi, im doing an app with streamlit and

ggingerwizard
·
Hi, im doing an app with streamlit and llama index and im curious what the recommendation is re which objects to cache and how. What level of thread safety is guaranteed? should i share indexes or engines across requests or create per user session?
L
2 comments
LLogan M
hmm, in general, anything thats writing to an object in memory is probably not thread safe. Reads should be fine as far as I know
LLogan M
If you have a chat engine/agent for example, that has memory, that won't be thread safe
Add a reply
Sign up and join the conversation on Discord