Hi folks, I'm running into an issue with

rrichard

Hi folks, I'm running into an issue with using Llama Index to query from my Postgres Vector Store.

I used llama index to ingest a lot of data into a self-hosted Postgres DB with pgvector enabled (~18M rows), using OpenAI embeddings.

When I try to run queries against this DB, I get search hits maaaybe 1% of the time. The other 99% of the time I get an empty response.

Then I created a separate PGVector container with a small subset of my data (~100 rows). Then I would get the expected results almost all the time as expected.

Are these results expected? I am not too familiar with how vector searches work under the hood, and these observations puzzle me. I would love some guidance from someone more knowledgeable than me. Thanks in advance!

23 comments

LLogan M

Any code sample for how you are running queries?

You can test the retrieval directly too

Plain Text

index.as_retriever(similarity_top_k=2).retrieve("query")

LLogan M

Pretty unexpected I'd say, but I'm also not familiar enough with postgres to explain why this is the case

LLogan M

(likely related to the size of the index?)

rrichard

Hi Logan, thanks for the reply. Here is our code that makes the query:

Plain Text

# Get the vector search to be confined on the appropriate documents
    filters = [
        MetadataFilter(key="doc_id", value=document_ids, operator=FilterOperator("in"))
    ]
    vs_index = manager.get_vs_index()
    query_engine = vs_index.as_query_engine(
        llm=sllm,
        similarity_top_k=similarity_top_k,
        filters=MetadataFilters(filters=filters),
    )
    response = await query_engine.aquery(query_str)

rrichard

Plain Text

manager.get_vs_index():



def get_vs_index(self) -> VectorStoreIndex:
        return VectorStoreIndex.from_vector_store(
            self._vector_store, self._embedding_model
        )

rrichard

maybe, I haven't seen anything online about it at all though.

LLogan M

Could it be that your filter is filtering out everything? I imagine it would work every time no matter what without the filter

rrichard

I can try that, however there are no problems with the filter when I query against the DB that only contains 100 rows

rrichard

@Logan M do you have any insight into how the query is constructed?

rrichard

ie are the filters applied in the same query as the similarity search?

LLogan M

The code is all here
https://github.com/run-llama/llama_index/blob/b789559b23402118bfd05d65dc65ff2192817866/llama-index-integrations/vector_stores/llama-index-vector-stores-postgres/llama_index/vector_stores/postgres/base.py#L829

rrichard

Thanks @Logan M . It seems to be that it is constructing a query that includes the filtering and similarity search in the same query. Is there a way within llama index to split this into two steps.- first get back nodes and then do a similarity search only on those nodes?

LLogan M

I'm not sure what you mean 🤔 Thats what the filtering is doing already

I think I mentioned it above, but you can call the retriever directly to get the nodes

nodes = index.as_retriever(similarity_top_k=2, filters=filters).retrieve("query")

rrichard

Oh shoot, I missed that message entirely. Will try it out, thanks!

rrichard

apologies for the additional questions. I tried querying with the retriever directly and I dont think that's what I'm looking to do.

To clarify my use-case, I have a pgvector table containing 18M rows from all of my documents. What I want to do is, given an document ID and input embedding, find the most similar rows from within a single document.

So what I want to try now is to run 2 queries.
First I want to pull all the nodes where document_id matches the input document_id

Then, out of those nodes, I want to run a similarity search and find the most similar node.

Is this something I can do in Llama index? Thanks again in advance!

LLogan M

There is a method get_nodes() in the base vector store class, but its not implemented for the postgres vector store (yet)

LLogan M

PRs very welcome!

rrichard

@Logan M - say no more 😉

https://github.com/run-llama/llama_index/pull/16026

LLogan M

Great work!

rrichard

Hi @Logan M , sorry to keep asking questions here, but how does one query only within a set of nodes using Llama index?

Now that I can use PGVectorStore to get the nodes matching a document_id directly, I want to do a query and find the top k within those nodes only.

Here is my code. It seems to be trying to insert the nodes and then doing a broad query against the entire table, which is not what I expect.

Plain Text

vs_index = VectorStoreIndex(
        nodes,
        embedding_model=manager.embedding_model(),
        storage_context=storage_context,
    )
    query_engine = vs_index.as_query_engine(
        llm=sllm,
        similarity_top_k=similarity_top_k,
    )
response = await query_engine.aquery(query_str)

Thanks for all the help so far!

LLogan M

Ideally, the metadata filters would just do this for you when you call query 😅

But yea, to do it this way, don't include the storage context here

LLogan M

It'll just make a temp in-memory index

rrichard

wonderful, i think that worked. thanks so much for all the help here @Logan M , really appreciate your support.

as for why the metadata filters aren't doing exactly this -- my suspicion is that this is a result of how pgvector decides to resolve the query 🤷‍♂️

Add a reply

Find answers from the community

Hi folks, I'm running into an issue with