Find answers from the community

Updated 2 months ago

Parallel query pipeline issue

I'm trying to troubleshoot an issue with my query pipeline. I'm trying to run the join on the output of worker_llms (llms run in parallel) but the join always seems to run right after worker_query and before the worker_llms. When I visualize the DAG it looks correct.

Plain Text
def get_judge_engine() -> QueryPipeline:
    # Setup LLMs
    judge_llm = Settings.llm
    worker_llms = {}
    num_workers = int(os.getenv("NUM_WORKERS", "5"))
    for i in range(num_workers):
        worker_llms[str(i)] = Settings.llm
    
    # Construct the query pipeline
    p = QueryPipeline(verbose=True)

    # Define the pipeline nodes.
    module_dict = {
        **worker_llms,
        "worker_query": WORKER_PROMPT_TMPL,
        "judge_query": JUDGE_PROMPT_TMPL,
        "llm_judge": judge_llm,
        "join": ArgPackComponent(),
    }
    p.add_modules(module_dict)

    # Add links between nodes 
    for i in range(num_workers):
        p.add_link("worker_query", str(i))
        p.add_link(str(i), "join", dest_key=str(i))

    p.add_link("join", "judge_query", dest_key="context_str")
    p.add_link("judge_query", "llm_judge")
   
    # Generate visualization
    net = Network(directed=True)
    net.from_nx(p.dag)
    net.save_graph("rag_dag.html")

    # Return the final pipeline.
    return p
A
L
7 comments
@Logan M here’s what my DAG looks like when I visualize it
Also here’s an excerpt of what I see in the debug. You can see the join firing earlier than expected

Running module worker_query with input:
Original query goes here

Running module join with input:

Running module 0 with input:
messages:
Act as a reasonably strict expert ...

Running module 1 with input:
messages:
@Logan M I did more testing and it works through version 1.10.46. Is this a bug I should ticket or were there just some breaking change I need to make updates for?
You mean it stopped working after v0.10.46? Or it didn't work before that?
It stopped working in version 1.10.47
Hmmm, must be a bug 👀
Add a reply
Sign up and join the conversation on Discord