Find answers from the community

Updated 3 months ago

hello @Logan M ! sorry to bother you. is

hello @Logan M ! sorry to bother you. is it possible that the reranker is skipped using the chat_engine on a query_engine and memory_buffer with context mode ? I am using cohere as the reranker, as specified in the example notebook.I checked the use of the api key in the Coehere dashboard and it doesn't seem to receive calls.


 # Define the QueryFusionRetriever
retriever = QueryFusionRetriever(
[vector_retriever, bm25_retriever],
similarity_top_k=20,
num_queries=4,
mode="reciprocal_rerank",
use_async=True,
verbose=True,
)

# Define the query engine with the reranker as a postprocessor
query_engine = RetrieverQueryEngine.from_args(
callback_manager=callback_manager,
retriever=retriever,
response_mode="compact",
verbose=True,
node_postprocessors=[reranker]
)

memory = ChatMemoryBuffer.from_defaults(chat_history=messages or [])

## Define the Chat engine on Quey engine with memory buffer and context mode as query
chat_engine = index.as_chat_engine(
query_engine=query_engine,
chat_mode="context",
memory=memory,
verbose=True
)
W
c
18 comments
Context chat engine does not take query_engine , it takes in a retriever.

So it forms retriver on its own if not given hence your query_engine is not being used,

Try with these changes and see if its working for you:
Plain Text
 chat_engine = index.as_chat_engine(
            retriever=retriever, # Your defined retriever
            chat_mode="context",
            memory=memory,
            verbose=True,
node_postprocessors=[reranker]
        )
thanks for the reply, i figured it wasn't going. i tried editing as suggested but i get this:

TypeError: RetrieverQueryEngine.from_args() got multiple values for argument 'retriever'.
Can you share the full code once please
sure! @WhiteFang_Jr
Yes the code looks right now, can you share the full traceback after running this once
oh I see it first creates a query_engine, but we dont need that. You can do this:
Plain Text
from llama_index.core.chat_engine import ContextChatEngine
chat_engine = ContextChatEngine.from_defaults(retriever=retriever, node_postprocessors=[reranker],memory=memory,verbose=True)

once you have the retriever create the chat_engine instance like this
I believe this will work
It works perfectly ! Thank you very much !!!

full log with CBEvenType.Rerank (❤️ ):

[15:44:32] INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
[15:44:33] INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
[15:44:45] INFO HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" _client.py:1038
**
Trace: chat
|_CBEventType.RERANKING -> 1.730763 seconds
|_CBEventType.SYNTHESIZE -> 10.455676 seconds
|_CBEventType.TEMPLATING -> 0.0 seconds
|_CBEventType.LLM -> 10.436721 seconds
**
I take advantage of your kindness one last time. the log I sent before I was using voyage ai as the provider of the reranker while here I use cohere:

[15:50:54] INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
[15:50:55] INFO HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK" _client.py:1038
[15:51:04] INFO HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" _client.py:1038
**
Trace: chat
|_CBEventType.RERANKING -> 0.719348 seconds
|_CBEventType.SYNTHESIZE -> 9.49498 seconds
|_CBEventType.TEMPLATING -> 0.0 seconds
|_CBEventType.LLM -> 9.463584 seconds
**


apparently for using VoyageAI the HTTP request response marking OK for using the service does not arrive while for Cohere it is present. Both the logs show the CBEvenType.RERANKING.
Does this however mean that it has passed or that VoyageAI reranking may not have happened despite the CBE?
Checking on the VoyageAI dashboard, it looks like they update it once a day. So I may as well check tomorrow in case.
It should show the log atleast tbh. Not sure why it didnt show atleast the http request log in the above request
ok. tomorrow i will check if the VoyageAI dashboard show the calls and post the result here, if any 😅. it would be interesting to understand why VoyageAI behaves differently. thank you very much for the support !!!
Hey, it seems that the API call for reranking with VoyageAI works, it's just not displayed in the request.
Attachment
image.png
I see 👀 but anyway it worked that was the target 😆
yes yes, that's good. at least for those like me who went crazy over it a few days now will know. thanks again for yesterday.
Add a reply
Sign up and join the conversation on Discord