Can you point me to the author of the

eerlebach123

Can you point me to the author of the following demo on LlamaIndex: Query Pipeline Chat Engine (https://docs.llamaindex.ai/en/stable/examples/pipeline/query_pipeline_memory/)? The demo does not work for me: the pipeline states that it only accepts a single output. Apparently, there i an error in lower-level libraries:

Plain Text

File ~/src/2024/llama_index_gordon/basics/.venv/lib/python3.12/site-packages/llama_index/core/query_pipeline/query.py:410, in QueryPipeline.run(self, return_values_direct, callback_manager, batch, *args, **kwargs)
    406     query_payload = json.dumps(str(kwargs))
    407 with self.callback_manager.event(
    408     CBEventType.QUERY, payload={EventPayload.QUERY_STR: query_payload}
    409 ) as query_event:
--> 410     outputs, _ = self._run(
    411         *args,
    412         return_values_direct=return_values_direct,
    413         show_intermediates=False,
    414         batch=batch,
    415         **kwargs,
    416     )
    418     return outputs

Thanks.

13 comments

WWhiteFang_Jr

Can you add the entire error trace, that will be helpful

LLogan M

Seems like it flat out doesn't work anymore 😥 Might just delete the example, I don't think query pipelines are going to get support for much longer

LLogan M

the code is so complex too 😅 Working on something easier to use and maintain

eerlebach123

That is good to know, Logan. Are the official examples run with integrated tests? That would be very useful. What might replace query_pipelines? I had a discussion with Perplexity (AI) to investigate imitating some aspects of the LCEL language in Langchain, at least for single input/output pipeline elements. Even for multi inputs/outputs, there should be a more elegant approach based on operators. Is anybody looking into that?

eerlebach123

@WhiteFang_Jr : here is the full error message:

Plain Text

(see the file "message.txt"

I also attach the notebook I am running, so you can check against whatever you have.

LLogan M

There are a bunch of unit tests for pipelines, not this specific example.

Imo LCEL has been pretty poorly recieved by the community. And I think the problems it has are similar to query pipelines

extremely hard to debug (if you make a connection issue, or a component breaks, it can be hard to figure out where in your pipeline it broke)
can't (easily) do stepwise execution or set break points and logging
hides the actual implementation of what a component does
more-so with query pipelines, but the syntax is pretty hard to read. Once you have a complex pipeline, it can be impossible to know what its doing without drawing it

We are working on an event-driven approach instead

eerlebach123

Ok. Is there a way to stay informed as this work progresses rather than wait for it to complete? What do you recommend as an alternative to query pipeline? Standard procedural coding?
What is your opinion of DSPY?
I assume that any new implementaion will first be synchronous. I looked at the source code and I saw in def _run reference to asynchronous components, which is strange to me.

LLogan M

Hmm, there should be an annoucment next week actually.

Yea tbh I'd recommend standard imperitive coding for now actually.

DSPY is interesting, but the interface seems a little shallow. Like you can declare stuff in a pytorch like syntax, but beyond the prompt optimization stuff, it isn't really doing any heavy lifting for you. In comparison to actual pytorch, its handling a ton of huge things like auto-grad, backpropagation, etc.

(And the optimization stuff seems less helpful in my experience, ts just generating few-shot prompts for you.)

it references async because it handles batch inputs (which batches can be run async concurrently, so why not)

eerlebach123

Thank. I look forward to the announcement. The week is almost over. I thought DSPY is elegant, but I have not tried to optimize anything.

eerlebach123

I am still debugging. The following pipeline is fine.

Plain Text

pipeline = QueryPipeline(
    modules={
        "input": input_component,
        "rewrite_template": rewrite_template,
        "llm": llm_c,
        "rewrite_retriever": retriever,
    },
    verbose=True,
)

# run both retrievers -- once with the hallucinated query, once with the real query
pipeline.add_link(
    "input", "rewrite_template", src_key="query_str", dest_key="query_str"
)
pipeline.add_link(
    "input",
    "rewrite_template",
    src_key="chat_history_str",
    dest_key="chat_history_str",
)
pipeline.add_link("rewrite_template", "llm")
pipeline.add_link("llm", "rewrite_retriever")

If I add the following to the pipeline and add

Plain Text

pipeline.add_link("input", "query_retriever", src_key="query_str")

I get two outputs: # # Multiple outputs

Plain Text

["rewrite_retriever", "query_retriever"]]

Any ideas?

eerlebach123

Possible resolution:

Plain Text

@dosu, I think I found the error:

pipeline = QueryPipeline(
modules={
"input": input_component,
"rewrite_template": rewrite_template,
"llm": llm_c,
"rewrite_retriever": retriever, # <<<<<<
"query_retriever": retriever, # <<<<<<<
"join": argpack_component,
"reranker": reranker,
"response_component": response_component,
},
verbose=True,
)

Plain Text

Notice that the `retriever` value appears twice in the modules list. Is that allowed? It matches perfectly with the multiple outputs. Perhaps the retriever should have been two different instances of the Retriever class?

LLogan M

Interesting! I've actually had a similar issue in the past reusing an llm component. Didn't catch the retriever being used twice here

eerlebach123

I was wondering, as an aside, if you could share the branch where the approach that will supercede query pipelimes is located? I'd like to take a look if you don't mind. If not, I also understand. Thanks.

Add a reply

Find answers from the community

Can you point me to the author of the