Find answers from the community

Updated 2 months ago

For each query, I'd like to perform

For each query, I'd like to perform several custom vector retrievals...and control which nodes are selected for the prompt. Would it be best to sublcass VectorIndexRetriever to achieve this?

15 comments

BBioHacker

Depends on what you define as custom. You can already customize the nodes retrieved using response_mode=no_text and then filter through them. You can also perform multiple vector retrieval and merge the selected nodes together.
Is this the use case you are looking for?

ddoughboy

Yes, I just want to parse the sub topics within a query on my own and perform multiple vector retrievals.

ddoughboy

I do not wish to use SubQuestionQueryEngine.

ddoughboy

I'm not sure that response_mode=no_text would suffice here.

ddoughboy

Or perhaps I should bypass the query altogether and do something more fine grained?

LLogan M

You can use the retriever directly for this use case, and then use a response synthesizer on its own once you have your nodes

LLogan M

Plain Text

retriever = index.as_retriever(similarity_top_k=2)
nodes = retriever.retrieve("query")

from llama_index.core import get_response_synthesizer

synth = get_response_synthesizer(response_mode="compact", llm=;;m)

response = synth.synthesize("query", nodes)

BBioHacker

To add to this @doughboy you should consider a pre-processing step where you use a language model to extract out the topics in the query and then retrieve them for each topic. Then you can synthesize the response using Logan's example.
Also if these topics are keywords, I recommend BM25 or a hybrid retriever.

ddoughboy

I already have a sure-fire way to parse the topics without a language model that might hallucinate.

ddoughboy

So I should just do nodes = retriever.retrieve(topic) multiple times, and then it's business as usual?

BBioHacker

yeah you also need to merge them and then synthesize

ddoughboy

Will do. Thanks, guys!

ddoughboy

@Logan M @BioHacker If I'm not using a query engine, how do I incorporate a node postprocessor, such as MetadataReplacementPostProcessor?

ddoughboy

Do I simply do this?

Plain Text

nodes = postprocessor.postprocess_nodes(nodes) response = synthesizer.synthesize(prompt, nodes)

LLogan M

Exactly!

Add a reply