I'm using as_chat_engine to stream the chat. chat_engine = index.as_chat_engine(
streaming=True,
similarity_top_k=5,
chat_mode="condense_plus_context",
llm=llm,
response_synthesizer=response_synthesizer,
node_postprocessors=[SentenceEmbeddingOptimizer(threshold_cutoff=0.8)],
verbose=True,
system_prompt=(
"""Where possible use bulleted list and good formatting to help understand the content better. Please note to answer the questions within the context of the documents provided."""
),
)
response = chat_engine.stream_chat(query)
sources = []
for node in response.source_nodes:
sources.append(node.node.metadata["file_name"] + " (score="+ str(node.score) + ")")
I'm getting the response- The document does not have any specify answer.
but still getting the sources and scores 0.83, can anyone help me here how I'm getting the scores if documents does not have relevant answers ? I don't want to show the sources and scores if it doesn't have relevant answer.
You can try one approach, ask the LLM to add a unique label when it is not able to answer.
based on that label ( if you find that in you can remove the nodes from the context.
Second approach will be to compare the answer and the nodes and check the similiarity score for these if they pass the threshold then you show else you skip
