Troubleshooting interactive conversation issues with a new embedding model

Question

Hello, I'm using the context mode chat engine and am testing out a new embedding model (OpenAI's text-embedding-3-large). However, when I then run my chat engine, I am no longer able to have an interactive conversation. Can someone help me out?

WhiteFang_Jr · Answer

hey, would you mind sharing the code ?

Torsten · Answer

storage_context = StorageContext.from_defaults(persist_dir=f"{product_code}_llama") index = load_index_from_storage(storage_context) engine = index.as_chat_engine( chat_mode="context", verbose=True, system_prompt=prompt, )

Torsten · Answer

And these are my settings:Settings.llm = OpenAI(model = GPT_MODEL, temperature = 0.0, max_tokens = 3000)Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-large")Settings.chunk_size = 256Settings.chunk_overlap = 64

Torsten · Answer

How I create the chat_history:
prior_conv.append(
ChatMessage(
role=MessageRole.USER, content=START_CONTENT + question + END_CONTENT
)
)
prior_conv.append(
ChatMessage(
role=MessageRole.ASSISTANT, content=START_CONTENT + answer + END_CONTENT
)

WhiteFang_Jr · Answer

What error are you facing, can you share that as well

Torsten · Answer

No error, it just won't answer follow-up questions anymore, while my prompt didn't change

WhiteFang_Jr · Answer

You can check if follow up questions are bringing correct nodes or not

Logan M · Answer

I'm not sure what you mean by "it won't follow up" ?

Logan M · Answer

How are you calling the engine? Are you passing in that cht history list?

Torsten · Answer

Ok let's say I ask the chatbot how to upload data. Then it gives me a 5 step plan like step 1 clikck here step 2 do this, et cetera. Now let's say step 3 is a little unclear to me, then as a follow-up question I might ask: "Can you explain step 3 a little more clearly?". And this breaks with the new embedding.

Torsten · Answer

This is how I call the chat engine: response = query_engine.chat(question, chat_history=prior_conv)

Torsten · Answer

I'll try that, thanks!

Torsten · Answer

Should the final nodes used for the answer include the provided chat_history?

Find answers from the community

Troubleshooting interactive conversation issues with a new embedding model