I have a question regarding `query`

I have a question regarding query method. While creating the index, if I have some metadata like below:

nodes = []
for index, row in data_embedding.iterrows():
    text_embedding = json.loads(row["embedding"])
    text = row["combined"]
    node = TextNode(text=text, metadata={"uuid":row["uuid"], "brand_name": row["brand_name"]}, embedding=text_embedding)
    nodes.append(node)
    
index = VectorStoreIndex(nodes=nodes)
query_engine = index.as_query_engine(similarity_top_k=5)

While doing the quering using query method, do I have to pass metadata for this query as well or llamaindex will do this on its own in backend. I am really confused because I have seen a few videos where people were passing metadata explicitly and some were just defining the metadata in the TextNode but they didn't pass metadata in query. How exactly query method is working in backend with metadata?

5 comments

WWhiteFang_Jr

Adding metadata into a Node can help in many ways:

You can apply metadata filtering
Once nodes are retrieved, you can use some of the metadata for various reasons like showing URL as source.

When users are passing metdata while querying they must be applying filtering.

HHK

Thanks @WhiteFang_Jr for the answer;

so far example;
response = query_engine.query("2K Games")

in this case, .query() will just do the embedding similarity in backend regardless of having metadata in TextNode and index?

HHK

https://discord.com/channels/1059199217496772688/1239490203899396146/1239490207930253349

In this thread, it seems like there is no need to pass the metadata from users end and LLamaindex will apply metadata filtering on its own. Can you please explain this?

WWhiteFang_Jr

No it will comapre on text + what metadata you have allowed to be seen by the embed model

WWhiteFang_Jr

In this example in transformation process, Titles and questions based on data is created using llm and when user query is passed it is compared along with text and this newly created data.

But this metadata may not contain info like:

Maybe you want to add some url
Add some value specific data or maybe unique code to each node.

This you'll have to add by yourself

Add a reply

Find answers from the community

I have a question regarding `query`