let the llm decide.. maybe return a dict which includes both a topic key , and the user query string.
You parse the dict, throw the query string at the index search, then do filter by topic metadata (if the vector db supports this natively).
Another approach using llamaindex is to use tools and MetadataFilters:
In this approach, you define a vector_query tool , you pass it the actual user query string and the topic (infered by the LLM) as arguments.
something like
from llama_index.core.vector_stores import MetadataFilters
def vector_query(
query: str,
topic: str) -> str:
""" Perform vector search based on topic. Try to
infer topic category from user query.
query [str]: the user's question.
topic [str]: Filter by topic. Topic can be
sales, labor, technical support.
"""
metadata_dicts = [
{"key":"topic", "value": topic}]
query_engine = vector_index.as_query_engine(
similarity_top_k=2,
filters=MetdataFilters.from_dicts(
metadata_dicts,
)
)
return query_engine.query(query)
vector_query_tool = FunctionTool.from_defaults(
name="vector_tool",
fn=vector_query
)then pass the vector_query_tool to the LLM agent as a tool.
You can also augment the LLM prompt by explicitely asking it to infer the topic (if you don't just want to rely on the function description - as shown above- which is passed to the LLM prompt).
