Optimizing memory and performance for multi-user app with shared retrievers and rerankers
Optimizing memory and performance for multi-user app with shared retrievers and rerankers
At a glance
The community member is asking about the best approach for creating an app for multiple users, specifically regarding the use of retrievers and rerankers. They want to know if these components can be shared across multiple users, and if so, whether there would be any latency issues. The community member is looking for guidance on finding a good balance between sharing these components and maintaining individual user memory.
When creating an app for multiple users, obviously I need memory to be individual for each user, but what about retrievers and rerankers, and if I can use them for multiple users at the same time is there latency or not, and if there is latency what is in your experience good balance for sharing retrievers and rerankers
