@Logan M I needed a few clarifications regarding the Qdrant vector store integration with Llamaindex. :- I saw that we support the filtering feature on payload (metadata in documents) via the QdrantVectorStore class. I don't see anything about payload_index though. Also, what about the optimizers concept? How useful are these features and what is the likelihood of requiring them? Do you recommend using the QdrantVectorStore wrapper or implementing everything using the qdrant-client in my python app.
My thought process is that in future , if we switch from Qdrant to let's say Pinecone, the code changes would be very few, should I use the QdrantVectorStore abstraction since the implementation of filtering etc is kinda agnostic to the vector store. ? Am i assuming the right thing? Any suggestions and comments are welcome! Tagging you @Kacper Łukawski for your inputs as well!
In general I agree, switching vector stores is mostly painless when using llama index (except that migrating data from one db to another will be difficult and/or not possible :PSadge: ). Using the QdrantVectorStore wrapper instead of pure qdrant gives you access to all our other stuff too (different query engines, agents, etc.)
The sentence embedding optimizer just tries to remove sentences that aren't helpful from the retrieved nodes. This saves token costs (and can speed up response times), but can possibly also degrade answer quality in some cases. We are looking into ways to exapnd this feature.
By payload index I mean this - Payload index in Qdrant is similar to the index in conventional document-oriented databases. This index is built for a specific field and type, and is used for quick point requests by the corresponding filtering condition. Basically doing client.create_payload_index().
Do we have plans for integrating this as well in future?
I didn't get this entirely - Using the QdrantVectorStore wrapper instead of pure qdrant gives you access to all our other stuff too (different query engines, agents, etc.)
why is access to diff query engines, agents etc dependent on using the QdrantVectorStore wrapper?
