Yea it's a tricky thing. You could use a custom node-postprocessor and assign weights in there. You'd have to set the top_k pretty high and then after applying weights, filter down to a smaller top_k
thats what im currently looking at now! My only slight confusion is around how i use the node postpocessor when using the index purely as a receiver?
Also does increasing top_k increase the cost of querying the index? or doenst it matter if you filter it down before passing it through as context?
And one last thing is if im using the index purely as a retriever, do i need a custom post processor or could i just use it as a retriever with a high k value and then filter after instead?
Hmm, you are using the index to only retrieve nodes, like with "no_text" set? You can definitely just filter them after you get the nodes in response.source_nodes
Increasing the top_k does not increase costs, as long as you filter the nodes down before sending them to synthesize a response 🙂
