Find answers from the community

Updated 2 months ago

Vectors

I'm using chat engine to query a vector store. If no vectors exist in the vector store I dont get an error, which is a problem. It will simply send the query to the LLM with an empty context. How do I avoid this?

18 comments

LLogan M

What chat engine are you using?

(Probably you can also check the vector store for content beforehand too)

kkevingoed

@Logan M

Plain Text

index.as_chat_engine(
        chat_mode=ChatMode.CONTEXT,
        condense_question_prompt=PromptTemplate(CHAT_PROMPT_TEMPLATE),
        chat_history=chat_history,
        agent_chat_response_mode="StreamingAgentChatResponse",
        similarity_top_k=20,
    )

kkevingoed

i would have expected it to throw an error if there's no vectors at all

LLogan M

Yea, here it would retrieve 0 vectors, leaving the system prompt without context, and the LLM would respond otherwise. Feel free to open a PR to change that behaviour
btw, setting top-k=20 will probably break most LLMs in this chat mode, unless your chunks are small, or your LLM has a huge context window