Find answers from the community

B
Blake
Offline, last seen 3 months ago
Joined September 25, 2024
B
Blake
·

issue

issue:
query() returns id when using simplevectorstore
but when faissvectorstore, query() returns ref_doc_ids

Q: how can i retrieve nodes when using a faiss query - e.g. using the returned ref_doc_ids? (e.g. get_nodes does not work w ref_doc_id - is there another func?)

Context: the vectordir was created using the follwoing code:
32 comments
L
B
B
Blake
·

Similarity

I have these embeddings that i extracted from my index and i want to query to get the cosine similarity from another llama index (without using a query str/text str) - how do u recc taking raw embeddings and getting the most similar from an index?

eg: QueryBundle type only takes mandatory query_str

feels like im missing a simple way to do this
23 comments
B
L
Any way to get the raw embeddings of retrieved nodes (not just indices + similarities as pictured) when using faiss vectorstore?
Or a hack/workaround to get the raw embeddings here?

example of my current retrieval code:

Plain Text
    from llama_index.vector_stores.types import VectorStoreQuery
      vsq = VectorStoreQuery(query_embedding=item.query_embedding, similarity_top_k=item.num_results, output_fields=['embeddings'])
      query_result = index.vector_store.query(query=vsq)
      return query_result
    

and:

Plain Text
 x = await index.as_retriever(similarity_top_k=num_results_to_retrieve).aretrieve(QueryBundle("not used", embedding=item.query_embedding))



Looking make clusters from my index, and to create the clusters i need the raw embeddings
3 comments
B
L
Recommended way to retrieve the most dissimilar nodes? i.e. the least similar

(I'm using Faiss as my vectorstore)
2 comments
B
L
B
Blake
·

Refresh

RE: updating an index

Is there any method for inserting docs into an existing/loaded index that compares new vs. existing doc hashes to prevent re-embedding? Or any other mechanism

Seems index.refresh() doesn't work like I thought it did
19 comments
B
L
B
Would love to see an example of integrating with llama index w Arize/phoenix:

https://colab.research.google.com/github/Arize-ai/phoenix/blob/main/tutorials/llama_index_search_and_retrieval_tutorial.ipynb#scrollTo=HXL7rQms5vL-

without using a pre-mande query_data_complete3.parquet

and without using query_engine.query(query) for the LLM calls (would want to use custom llm calls - openai func calls specifically)
2 comments
B
A
Can anyone explain the 'custom_embedding_strs' in the new querybundle feature?

I don't quite get what 'list of strings used for embedding the query' means.
21 comments
B
d