Hi guys I'm a noob and was wondering if

SSattyMan

Hi guys I'm a noob and was wondering if there any way to get the chunkid for chunks in the document. I'm applying security rules to show relevant facts to the user and filter out the rest.Also could somebody point me to the APi doc to Llama-Index. I cant seem to find it. TIA.

9 comments

WWhiteFang_Jr

Every doc has a DocID, You can let it be set on its own or if you want to set the doc_id, You can do that as well.

Plain Text

from llama_index import Document

doc = Document(text="This is first doc",doc_id="Set_YOUR_OWN_ID" )
print(doc) # It will print the doc text and the created doc ID for this.

# you can change the default created doc_id as well
doc.doc_id = "Set_YOUR_OWN_ID"

Docs link for LlamaIndex: https://docs.llamaindex.ai/en/stable/index.html

SSattyMan

Thanks @WhiteFang_Jr . Whats the API call to retrieve it?

WWhiteFang_Jr

There's no API call for this. You can directly do the above step in a python script.

SSattyMan

Sorry should have clarified.
When i get the response back from this line of code:
response = query_engine.query("What did the author do growing up? ")
In addition to the chunck of text that gets returned I also need the chunckid. That way I can map it to the user id or the doc id to see if the user has access to the underlying doc. Makes sense?

WWhiteFang_Jr

Ah I see.

So the response is formed based on the context that is provided to the llm.

The context is provided in the form of nodes which are formed on your custom data.

In the response object, you also get the nodes that have been used to help llm to generate the response.

You can access them like this.

Plain Text

for node in response.source_nodes:
   print(node)

WWhiteFang_Jr

These nodes have the ID. The response that is returned does not have the ID as it was generated by the llm

SSattyMan

Awesome!! Thanks for your prompt response @WhiteFang_Jr . One last followup question. Is there any place I can capture the actual doc name and the directory name from? Thxs

WWhiteFang_Jr

If it is a pdf file from which you have created the index. Then the filename and page no will be present in the extra info of your nodes

WWhiteFang_Jr

Also if you want to add additional info to some chunks like user id , roles , urls

You can add them into metadata of the nodes

Add a reply

Find answers from the community

Hi guys I'm a noob and was wondering if