Hey folks, on the topic of prompts, it

aamar

Hey folks, on the topic of prompts, it seems like my ServiceContext.from_defaults(query_wrapper_prompt="bla") is being ignored. Could there be something obvious I'm missing?

7 comments

LLogan M

query wrapper prompt only works with query engines (and should contain a string variable for {query_str}

Are you trying to use this in a chat engine? There's likely another method on the LLM you are using that we can leverage instead

aamar

I switched to query_engine yeah, chat engine was giving me too many problems. My full prompt looks like this (I copied if from the defaults):

Plain Text

    query_wrapper_prompt=
        "Context information is below.\n"
        "---------------------\n"
        "{context_str}\n"
        "---------------------\n"
        "Given the context information and general knowledge, "
        "answer the query.\n"
        "Query: {query_str}\n"
        "Answer: "

aamar

All I did is change "no prior knowledge" as a test

aamar

Do I need to do something with PromptTemplate beforehand maybe?

LLogan M

ohhh I think you are misunderstanding what query wrapper prompt is for.

I think what you intend to change are the RAG prompt template. See this page for example
https://docs.llamaindex.ai/en/stable/module_guides/models/prompts.html

https://docs.llamaindex.ai/en/stable/examples/customization/prompts/chat_prompts.html

aamar

Thanks! I'm assuming that I pass these templates to index.as_query_engine(). Do you know how I can find out what the arguments for that are? IDE has kwargs as Any

LLogan M

Yea this is because the kwargs get passed to both the underlying retriever and response synthesizer

With a vector index, this ends up accepting kwargs for both the VectorIndexRetriever and get_response_synthesizer function

https://github.com/run-llama/llama_index/blob/58344c46ca44c0e71a08335aeeec090349a47164/llama_index/indices/vector_store/retrievers/retriever.py#L21

https://github.com/run-llama/llama_index/blob/58344c46ca44c0e71a08335aeeec090349a47164/llama_index/response_synthesizers/factory.py#L29

tbh if you use the lower-level APIs, this will probably be a bit clearer. as_query_engine and as_chat_engine make it initially very easy, but the lower-level API offers clearer visibility into whats going on

https://docs.llamaindex.ai/en/stable/understanding/querying/querying.html#customizing-the-stages-of-querying

Add a reply

Find answers from the community

Hey folks, on the topic of prompts, it