Hi guys has anyone know how can we use

At a glance

Hi guys has anyone know how can we use advanced RAG with chat_engine? as I noticed i can do lots of advanced RAG with query engine, but not chat_engine. Or equivalently how easy is that to convert an advanced query engine to a chat engine?

16 comments

WWhiteFang_Jr

Its very easy tbh, Whatever you were doing when initialising your query_engine, just need to do the same for chat_engine

Plain Text

# This helps you to create query_engine
query_engine = index.as_query_engine(similarity_top_k=5,...)

# This allows you to create chat engine
chat_engine = index.as_chat_engine(similarity_top_k=5,...)

mmaryam

no actually, if i want to send retreiver to chat engine it doesnt accept, any advanced retriever or postprocessing is not easy to do

LLogan M

if i want to send retreiver to chat engine it doesnt accept im not sure what you mean, we have chat engines directly for retrievers

Plain Text

from llama_index.core.chat_engine import CondensePlusContextChatEngine

chat_engine = CondensePlusContextChatEngine.from_defaults(
  index.as_retriever(),
  llm=llm,
  system_prompt="...",
  node_postprocessors=[...],
)

mmaryam

thanks Guys

mmaryam

so @Logan M are u saying all the advance RAG techniques that i am able to do with query engine I can also do with CondensePlusContextChatEngine? so no restriction or differences between the two?

mmaryam

since at the end of the day i need a chat bot that can have the chat history

WWhiteFang_Jr

Yes, just pass in everything that pass while creating the query_engine

WWhiteFang_Jr

chat engine will have that

mmaryam

i try to undesrtand the differences of when i say index.as_chat_engine and when i use this CondensePlusContextChatEngine. I see their type is different.

mmaryam

the reason i am asking is that i am using this in a streamlit app, and when i have index._as_chat_engine its working fine and user can ask multiple questions no problem, But when i try the CondensePlusContextChatEngine to be able to have more advanced retrieval stuff, when user asks the first question is fine, but second question it looks like not working fine!

LLogan M

as_chat_engine is a shortcut to many chat engines, CondensePlusContextChatEngine is one of them. Personally I think its the most controllable and configurable, and the best balance of speed

if you don't put a chat_mode when doing as_chat_engine like index.as_chat_engine(chat_mode="..."), it will default to an agent

mmaryam

yes, now that i checked my code, I have passed the chatmode already and so both of them (1)using index.as_chat_engine or using CondensePlusContextChatEngine) have the same output shape, so not sure why the streamlit just take one of the questions and not let the second question run!

mmaryam

here is the way i am defining my chat_engine: "if "chat_engine" not in st.session_state.keys(): # Initialize the chat engine
st.session_state.chat_engine = advance_rag_chat_engine(index)"

mmaryam

do you know why might be the problem?

mmaryam

and this is inside my advance_rag_chat_engine : chat_engine = CondensePlusContextChatEngine.from_defaults(
retriever,
node_postprocessors = [colbert_reranker,llm_rerank_postprocessor],
)

mmaryam

i now know most probably its due to strealit function calls and cache https://docs.streamlit.io/develop/concepts/architecture/caching

Add a reply

Find answers from the community

Hi guys has anyone know how can we use