Find answers from the community

C
Chill
Offline, last seen 4 months ago
Joined September 25, 2024
Hey all, I'm trying to build out a simple rag QueryPipeline, but when I look at my traces in Phoenix it looks like the llm is being invoked twice?

My code:
pipeline.add_modules(
{
"llm": llm.as_query_component(),
"retriever": retriever,
"formatter": formatter,
"prompt_tmpl": CITATION_RAG_PROMPT.as_query_component(
partial={
"chat_history_str":chat_history_str,
"query_str":query,
"json_prompt_str":json_prompt_str
}
),
"output": output_parser
}
)

pipeline.add_link("retriever", "formatter", dest_key="nodes")
pipeline.add_link("formatter", "prompt_tmpl", dest_key="context_str")
pipeline.add_link("prompt_tmpl", "llm")
pipeline.add_link("llm", "output")

I'm guessing this is because the llm is a chat model, and it is converting my prompt input into a user chat message. Is this actually calling the LLM twice or is it just doing that prompt_to_messages conversion? And if it is calling twice, how can I just get the llm to call complete instead of chat when run through the QueryPipeline?
19 comments
C
L
a
When trying to import AzureOpenAIEmbeddings I am getting an issue cannot import name root_validator from llama_index.core.bridge.pydantic. This appears to happen in the source code of the class itself? The code snippet in question does say that it is not used, any advice?

llama-index: 0.10.62
llama-index-core: 0.10.62
6 comments
L
C