Gero

Hey everyone,

Hey everyone,

I'm currently working with Query Pipelines and I'm encountering an issue when passing a list of reranked nodes to a synthesizer. The problem arises when the entire list of nodes gets cropped in the prompt. I'm wondering if anyone has any insights into what might be causing this issue.

Plain Text

user_prompt = PromptTemplate("{query}")
reranker = CohereRerank(model="rerank-multilingual-v3.0", api_key=api_key, top_n=3)
response_synthesis_prompt = PromptTemplate(
    response_synthesis_prompt_str,
)
llm = OpenAI(model="gpt-4-turbo", system_prompt=system_prompt)

Plain Text

#define qp
p = QueryPipeline(verbose=True)
p.add_modules(
    {   
        "user_prompt": user_prompt,
        "router_retriever": router_retriever,
        "kg_retriever": router_retriever,
        "reranker": reranker,
        "response_synthesis_prompt": response_synthesis_prompt,
        "response_synthesis_llm": llm,
    }
)

Plain Text

p.add_link("user_prompt", "router_retriever")
p.add_link("user_prompt", "reranker", dest_key="query_str")
p.add_link("router_retriever", "reranker", dest_key="nodes")

p.add_link("user_prompt", "kg_retriever")

p.add_link("user_prompt", "response_synthesis_prompt", dest_key="query_str")
p.add_link("reranker", "response_synthesis_prompt", dest_key="nodes")
p.add_link("kg_retriever", "response_synthesis_prompt", dest_key="kg_nodes")
p.add_link("response_synthesis_prompt", "response_synthesis_llm")

I suspect that the length of the reranked nodes list may be exceeding the prompt size limits, leading to it being truncated with ellipses (...). However, I'm unsure how to address this issue effectively.

If anyone has encountered a similar problem or has suggestions on how to resolve it, I would greatly appreciate any advice or guidance you can provide.

Thanks in advance for your help!

3 comments

GGero

Hey there,

Hey there,
I hope you're doing well. I wanted to touch base regarding a little snag I've hit while working on our app.

Currently, I am working on an application where I have been utilizing the streaming feature. However, I have run into a significant hurdle: I cannot seem to get the streaming functionality to work as expected. Specifically, I am struggling to ensure that data streaming occurs correctly.

I have used other applications with similar versions where the streaming feature has worked flawlessly. However, in this particular application, I am unable to get it to function as expected.

To provide you with more context, I am employing RouterRetriever, a synthesizer, and a RetrieverQueryEngine in my implementation. I have configured the necessary attributes for data streaming (streaming=True), but unfortunately, it has not yielded the desired results. I have even tried the implementation in both Google Colab and Hugging Face, but I continue to face the same issue.

llama-index==0.10.20

Below is how I have implemented the function in my code:

response_synthesizer = get_response_synthesizer(text_qa_template=qa_prompt, streaming=True)

query_engine = RetrieverQueryEngine.from_args(
retriever=custom_retriever,
response_synthesizer=response_synthesizer,
node_postprocessors=[rerank],
streaming=True
)

respuesta = query_engine.query(pregunta)
respuesta.print_response_stream() # No funciona

I have also attempted to implement it this way in Hugging Face:

def responder(pregunta):
respuesta = query_engine.query(pregunta)

partial_message = ""
for text in respuesta.response_gen:
partial_message += text
yield partial_message # and it still doesn't work

I would greatly appreciate it if you could provide me with any guidance or advice on how to resolve this issue. I am willing to provide any additional information you may need to assist me in resolving this situation.

Appreciate your help!

5 comments

GGero

I created a knowledge graph index from

I created a knowledge graph index from nodes, but it's not working. It says it can't find relationships. When I create them using from_documents(), they work perfectly, but when I do it from nodes, it doesn't find the relationships. The graph seems to be created properly since I can visualize it, and the triplets are reflected in the database. Any ideas?

15 comments

Find answers from the community

Hey everyone,

Hey there,

I created a knowledge graph index from