I am looking to upgrade from Llamaindex

eelmegatan26

I am looking to upgrade from Llamaindex 0.7.6 to 0.7.13 and having a problem with the AI response from gpt-3.5-turbo-16k. I include a custom chat history interaction and a custom prompt that instructs the engine to only answer questions relevant to the index. When updating to 0.7.13 my custom instructions are largely ignored and the AI will answer questions not included in the index. This does not happen with 0.7.6.

Plain Text

chat_engine = index.as_chat_engine(
  chat_mode='react', 
  chat_history=custom_chat_history,
  similarity_top_k=max_top_k,
  refine_template=DEFAULT_REFINE_PROMPT, 
  text_qa_template=custom_prompt)

If anyone has some hints to hunt down why this happens it will be greatly appreciated.

5 comments

LLogan M

So if you install 0.7.6 right now today, it works, but then 0.7.13 it doesnt?

Nothing has changed in the react agent between those versions 🤔 Are you using openai? My guess is that openai has updated the model over time (which happens a lot, and always breaks prompts)

LLogan M

We have a full changelog here, nothing jumps out to me between those versions 🤔 https://gpt-index.readthedocs.io/en/latest/development/changelog.html#v0-7-6-2023-07-12

eelmegatan26

Thanks Logan, I will look into this

KKayla

Curious if you guys are able to figure this out. Would also like for it to only answer q's relevant to the index

LLogan M

If you want it to only answer questions relevant to the index, you may have better luck with a different chat engine. The context chat engine with a specific system prompt may work well
https://gpt-index.readthedocs.io/en/latest/examples/chat_engine/chat_engine_context.html

If you aren't using a chat engine, you may want to change the prompt templates for the query engine. While they already state not to use outside knowledge, some LLMs don't follow the current instructions always :PSadge:
https://gpt-index.readthedocs.io/en/latest/core_modules/model_modules/prompts.html#modules

Add a reply

Find answers from the community

I am looking to upgrade from Llamaindex