Hi Guys, As far as I understand we need to load an index every time to start querying against it. This process creates latency. So to resolve this, I am loading all indexes to memory first and then doing querying.is there any other way to do this? What if the number of bots in my system is dynamic?
Yea loading index while querying will create latency for sure!
You can use vector stores, and loading all the instance while starting your server will only create a connection point to each index. Thus not much memory will be needed.
Hey, thanks for the reply. So right now, I have been storing the loaded index instances of all indexes in memory while starting a server. How scalable is this process? Also, I want to give the users options to create new indexes, so the pausible way to do is to store an instance whenever a new index is created?
If the indexes are small in size and only going to remain in memory for a short period of time I think you can put it there only. But if you are laoding it back to back then using vector store would be a good way to move ahead.
what if I have 100k indexes, can I do some sort of caching? loading all indexes seems like a very naive way to tackle this problem. Can you suggest something else or share some documents on this side?
