hello @Logan M ! sorry to bother you. is

ccali94

hello @Logan M ! sorry to bother you. is it possible that the reranker is skipped using the chat_engine on a query_engine and memory_buffer with context mode ? I am using cohere as the reranker, as specified in the example notebook.I checked the use of the api key in the Coehere dashboard and it doesn't seem to receive calls.

# Define the QueryFusionRetriever
retriever = QueryFusionRetriever(
[vector_retriever, bm25_retriever],
similarity_top_k=20,
num_queries=4,
mode="reciprocal_rerank",
use_async=True,
verbose=True,
)

# Define the query engine with the reranker as a postprocessor
query_engine = RetrieverQueryEngine.from_args(
callback_manager=callback_manager,
retriever=retriever,
response_mode="compact",
verbose=True,
node_postprocessors=[reranker]
)

memory = ChatMemoryBuffer.from_defaults(chat_history=messages or [])

## Define the Chat engine on Quey engine with memory buffer and context mode as query
chat_engine = index.as_chat_engine(
query_engine=query_engine,
chat_mode="context",
memory=memory,
verbose=True
)

18 comments

WWhiteFang_Jr

Context chat engine does not take query_engine , it takes in a retriever.

So it forms retriver on its own if not given hence your query_engine is not being used,

Try with these changes and see if its working for you:

Plain Text

 chat_engine = index.as_chat_engine(
            retriever=retriever, # Your defined retriever
            chat_mode="context",
            memory=memory,
            verbose=True,
node_postprocessors=[reranker]
        )

ccali94

thanks for the reply, i figured it wasn't going. i tried editing as suggested but i get this:

TypeError: RetrieverQueryEngine.from_args() got multiple values for argument 'retriever'.

WWhiteFang_Jr

Can you share the full code once please

ccali94

sure! @WhiteFang_Jr

WWhiteFang_Jr

Yes the code looks right now, can you share the full traceback after running this once

ccali94

Attachments

WWhiteFang_Jr

oh I see it first creates a query_engine, but we dont need that. You can do this:

Plain Text

from llama_index.core.chat_engine import ContextChatEngine
chat_engine = ContextChatEngine.from_defaults(retriever=retriever, node_postprocessors=[reranker],memory=memory,verbose=True)

once you have the retriever create the chat_engine instance like this

WWhiteFang_Jr

I believe this will work

ccali94

It works perfectly ! Thank you very much !!!

full log with CBEvenType.Rerank (❤️ ):

[15:44:32] INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
[15:44:33] INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
[15:44:45] INFO HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" _client.py:1038
**
Trace: chat
|_CBEventType.RERANKING -> 1.730763 seconds
|_CBEventType.SYNTHESIZE -> 10.455676 seconds
|_CBEventType.TEMPLATING -> 0.0 seconds
|_CBEventType.LLM -> 10.436721 seconds
**

WWhiteFang_Jr

Awesome

ccali94

I take advantage of your kindness one last time. the log I sent before I was using voyage ai as the provider of the reranker while here I use cohere:

[15:50:54] INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
INFO HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK" _client.py:1786
[15:50:55] INFO HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK" _client.py:1038
[15:51:04] INFO HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" _client.py:1038
**
Trace: chat
|_CBEventType.RERANKING -> 0.719348 seconds
|_CBEventType.SYNTHESIZE -> 9.49498 seconds
|_CBEventType.TEMPLATING -> 0.0 seconds
|_CBEventType.LLM -> 9.463584 seconds
**

apparently for using VoyageAI the HTTP request response marking OK for using the service does not arrive while for Cohere it is present. Both the logs show the CBEvenType.RERANKING.
Does this however mean that it has passed or that VoyageAI reranking may not have happened despite the CBE?

ccali94

Checking on the VoyageAI dashboard, it looks like they update it once a day. So I may as well check tomorrow in case.

WWhiteFang_Jr

It should show the log atleast tbh. Not sure why it didnt show atleast the http request log in the above request

ccali94

ok. tomorrow i will check if the VoyageAI dashboard show the calls and post the result here, if any 😅. it would be interesting to understand why VoyageAI behaves differently. thank you very much for the support !!!

WWhiteFang_Jr

Sure sure

ccali94

Hey, it seems that the API call for reranking with VoyageAI works, it's just not displayed in the request.

Attachment

WWhiteFang_Jr

I see 👀 but anyway it worked that was the target 😆

ccali94

yes yes, that's good. at least for those like me who went crazy over it a few days now will know. thanks again for yesterday.

Add a reply

Find answers from the community

hello @Logan M ! sorry to bother you. is