Find answers from the community

Updated 3 months ago

Hello all, I have a RAG application that

Hello all, I have a RAG application that returns structured data over documents such as invoices and purchase orders. My aim is to retrieve 2 sets of information from those documents. 1) Header information like supplier details, customer details, document date, tax amounts, total amounts, etc. 2) Table information of line items like description, quantity, unit prices, sub-total amounts.

After I parse the document, I split them into nodes, chunk them and build a retriever engine with reranker on top of it. Somewhat similar to this example here: https://docs.llamaindex.ai/en/stable/examples/node_postprocessor/FlagEmbeddingReranker/

I am currently sending 2 user queries (one to retrieve header information and another table information). In each of those prompts, I am trying to query all the information that exist in either the header or table. In this case, does having the reranker in the retrieval engine makes a difference in retrieval accuracy?

For example I have 2 nodes for header information. One of the node contains the document date, and the other contains the total amounts. If in my query, I am asking for both information at the same time. Both nodes will be retrieve and then passed to the LLM for generation right? If that is the case, having the reranker seems redundant?
L
g
10 comments
yea reranking seems redundant there.

Generally reranking is useful for setting a large top-k, and then reranking and filtering down to a smaller subset
Just what I thought
thank you for reply! @Logan M
I remove the reranker model from my rag pipeline and begin to see weird behaviors 😟
hey @Logan M just wondering if you would know if there is any guidelines/rules/recommendations that mention anything about the optimal similarity_top-k to set for the query engine and n for reranker? Understand that this may be based on the data being indexed and the amount of answers that one will be retrieving. For example, if I have an unknown amount of line items that I want to retrieve from tables that would span an unknown amount of pages. I would image as the retrieval gets larger, the higherthe number of k and n?
I think in general, the top-k can be as high as you want. But id probably limit n for reranking to limit how many LLM calls are needed to generate a response.

Higher top-k also means the reranking will be a bit slower though.

Its probably best to write some test cases and try to tune these params for your data
I'm running my own tests. Having trouble retrieving line items from table that exist in other nodes. I have around 20 line items spanning across a 4 page document, resulting in 4 object nodes. All of the should be retrieved but currently, only line items in one of the node is being retrieved. So I'm not sure if the top-k or n being set too low.
Any reason not to combine table that into a single object ?
Absolutely no reason at all. How can I combine them?
I was even thinking to do it from the markdown results coming from LlamaParse. Then converting them into nodes. But if tables can be combined into a single object, then it works too right?
Add a reply
Sign up and join the conversation on Discord