Find answers from the community

Updated 2 months ago

@Logan M I'm using vector store index

I'm using vector store index with hybrid search using qdrant and the process is running on a google cloud run, rather than have the ONNX model load in memory and redownload every time it boots up i wanted to save the model to a volume drive i mounted to the cloud run instance. Is there a way to load and save the sparse embedding generator to the volume mount (and also have it load the model from there)?

4 comments

LLogan M

I think you'd have to customize the function that loads the sparse model so that it looks in the location that you've saved the model

LLogan M

These two functions can be passed in
https://github.com/run-llama/llama_index/blob/83c8e78f51a06430588beb935bea448239c5a106/llama-index-integrations/vector_stores/llama-index-vector-stores-qdrant/llama_index/vector_stores/qdrant/base.py#L133

And the default for both those function hooks is here
https://github.com/run-llama/llama_index/blob/83c8e78f51a06430588beb935bea448239c5a106/llama-index-integrations/vector_stores/llama-index-vector-stores-qdrant/llama_index/vector_stores/qdrant/utils.py#L67

LLogan M

seems like you could just specify the cache_dir

LLogan M

Plain Text

sparse_fn = fastembed_sparse_encoder(cache_dir="./cache")

vector_store = QdrantVectorStore(..., sprase_doc_fn=sparse_fn, sprase_query_fn=sparse_fn)

Add a reply