Namespace

At a glance

The community member is asking about the recommended way to separate different "namespaces" with PGVectorStore, similar to how different indexes were used in PineCone to limit replies to a particular document/scope. Another community member suggests that using new namespaces/tables can get out of hand quickly, and recommends using metadata filtering to "namespace" the data in a single collection/table instead. Another community member proposes the idea of extracting keywords during ingestion and using them to improve the quality of generic document summaries, which another community member thinks is a good idea.

kkevingoed

What's the recommended way to separate different "namespaces" with PGVectorStore. In PineCone we used different indexes for different documents to limit the replies to a particular document/scope. I was wondering what the equivalent would in PGVectoreStore. Do you recommend setting up a new table for each "context"/"namespace"?

3 comments

LLogan M

New namespaces/tables can get out of hand a little quickly. It's fine in small numbers, but for larger stuff I would just use metadata filtering to "namespace" the data in a single collection/table

kkevingoed

@Logan M I have a question that isn't directly related to this. We had the idea to extract the keywords with the KeywordExtractor during ingestion and then if someone ask for a generic summary (e.g. "Summarize this document for me") we would go into the DB and get all keywords and attach them to the query. (e.g. "Summarize this document for me. Make sure that the summary covers all of the following keywords: [INSERT KEYWORDS]. Is this a good idea, or do you see any better ways to make sure that a request for a generic summary gets good quality results? 🙂 Our idea was that if we fetch top_k 10 results from the vector db with the keywords we would get way better results.

LLogan M

That sounds like a pretty good idea for writing summaries!

Add a reply

Find answers from the community

Namespace