Hello - have been building with llama

tthomoliverz

Hello - have been building with llama index for a while but am still beginner level coding/ engineering

Right now I have a script that loads data from Airtable, builds a Vector Store Index over them, and then which creates a chat engine for a user to ask questions over the data. The data is a series of quotes/ advice about productivity. User asks e.g. how can I sleep better and gets response based on quotes by several people.

There are a few things I am now looking to improve, specifically:

I want to have the index stored so that when a user asks a question, the response from their pov is quicker - I understand I can do this relatively easily using Llama Index or another store e.g. chroma (but could I not just use default? & is it possible to have the index constantly loaded so that when a user queries the index doesn't have to be built again?)

Users can also add to the database, so I also want to be able to refresh the index regularly or when an action is taken by a user - is there a classic way to do this?

I would like to allow users to apply some filters to their queries & then either load specific data based on those filters or retrieve only some data based on those filters - when is the right moment to do this? I will input some UI features for users to select different filters in the front end.

Grateful for any help/ advice

3 comments

WWhiteFang_Jr

For storing, How are you doing it currently like loading the index when a new query is asked?

i would suggest you keep the index in memory when your server gets started. Or use Qdrant in-memory. It will load the indexex in your memory [ nothing will leave your memory ]

This will completely depend on your use case, for example let say your bot provides an answer and your user is not satisfied with the answer so you give a feature using which user can update the answer. Based on this you can simply add this new info into index.

Filters also depended on your use case.

tthomoliverz

thanks - ok perfect. so load the index in mem when server gets started.

& then users ask queries to the already loaded index.

presumably there is a way to reload data & re-index regularly?

WWhiteFang_Jr

If the index is huge I would suggest using vector store like Qdrant!
Then you dont need to reload, just need to form the connection with the store.

Also for re-indexing, if you have new data, Index that only. No need to index everything again

Add a reply

Find answers from the community

Hello - have been building with llama