I think you are trying to tackle 2

He can build an agentic pipeline that maps tools to a specific intent, when and whenever that intent is expressed. He can't always narrow the search at the very beginning because customers explore and change intents dynamically based on observations.

@AudetesFortunaIuvat yes but without narrowing down the results, I can't send everything to the LLM as the context will be too long and recall will be low as well. Understanding the customer needs is important for us as we need to ask a few follow-up questions before doing RAG search but those follow-up questions should be derived from the products data after narrowing the search results.

@Roland Tannous with agentic pipeline, I can add X number of tools but what I have seen with agents is that because of the thought process and actions, agents take a bit more time and for us if the response time is more than 5 seconds then its not OK.

well it also depends on your frontend UX

so if you're not "streaming the experience"

then yes the user will notice the delay

but if it's something like sequential screens .. then you're giving the backend time to process

so basically divide the UX into multiple touchpoints

maybe even use a recomemndation system in parallel?

Here's a flow:

I ask about a chair .. maybe instead of just throwing the whole explanation in the agent... the agent returns top 10 matching product page links and the frontend processes those. the user opens one of them. The agent tab is still open.... That page, like a standard e-commerce page, holds all the variants available in terms of colors and sizes, then you've offloaded the retrieval of that information off the agent pipeline.

The page can also have a recommendation section and a section with tell me how to build this.. once the user clicks it.. then the agent does two things: grabs the information on the page, with the different variants sizes, colors.. then asks a series of questions to the user about those variants/variables... whcih size do you ? hwich color..
once the user enters this information, then the agent can go and figure out how to build it based on the product page information, the user answers and the intent...

By doing it this way , you're offloading some of the work to more traditional UX elements but also delaying the overloading of the agent with too much information "all the time"

so it's like AI assisted shopping

That approach sounds good to me. Which agent do you think will be good for this? I have used ReACT agent from llamaindex and so far I got good results apart from being slow.

react might be suitable.. it really depends what type of capabilities are you planning to give ur agent, you might go for a stacked agent architecture

yes python is already not the fastest interpreted language to start with

but are you sure it's not the latency on the LLM call? what LLM are you using