Find answers from the community

Updated 4 days ago

Troubleshooting interactive conversation issues with a new embedding model

Hello, I'm using the context mode chat engine and am testing out a new embedding model (OpenAI's text-embedding-3-large). However, when I then run my chat engine, I am no longer able to have an interactive conversation. Can someone help me out?
W
T
L
13 comments
hey, would you mind sharing the code ?
storage_context = StorageContext.from_defaults(persist_dir=f"{product_code}_llama") index = load_index_from_storage(storage_context) engine = index.as_chat_engine( chat_mode="context", verbose=True, system_prompt=prompt, )
And these are my settings:
Settings.llm = OpenAI(model = GPT_MODEL, temperature = 0.0, max_tokens = 3000)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-large")
Settings.chunk_size = 256
Settings.chunk_overlap = 64
How I create the chat_history:
prior_conv.append(
ChatMessage(
role=MessageRole.USER, content=START_CONTENT + question + END_CONTENT
)
)
prior_conv.append(
ChatMessage(
role=MessageRole.ASSISTANT, content=START_CONTENT + answer + END_CONTENT
)
What error are you facing, can you share that as well
No error, it just won't answer follow-up questions anymore, while my prompt didn't change
You can check if follow up questions are bringing correct nodes or not
I'm not sure what you mean by "it won't follow up" ?
How are you calling the engine? Are you passing in that cht history list?
Ok let's say I ask the chatbot how to upload data. Then it gives me a 5 step plan like step 1 clikck here step 2 do this, et cetera. Now let's say step 3 is a little unclear to me, then as a follow-up question I might ask: "Can you explain step 3 a little more clearly?". And this breaks with the new embedding.
This is how I call the chat engine: response = query_engine.chat(question, chat_history=prior_conv)
I'll try that, thanks!
Should the final nodes used for the answer include the provided chat_history?
Add a reply
Sign up and join the conversation on Discord