Find answers from the community

Updated 3 months ago

How can I get llama index to use the

How can I get llama_index to use the entire document i pass in as context
L
d
5 comments
If the document is too long, it will be broken into chunks

You can try increasing the chunk_size_limit though (the default is 1024)

service_context = ServiceContext(..., chunk_size_limit=200)
I think this is what you are getting at?
when prompting i mean
for one prompt i want it to use the entire context
Hmm maybe a longer explanation will help us get on the same page

When you insert documents, they are broken into nodes/chunks, as I detailed above

When using a vector index, each node/chunk is embedded

At query time, the query is also embedded, and the top k chunks are retrieved from the index (by default, the top k is two)

Using the default response mode (compact), all the text from the retrieved nodes, along with the query, is sent to the LLM

If the retrieved text is too long for the LLM to process, then it's broken into pieces and an answer is refined across multiple LLM calls

What's the exact issue you are running into then?
Add a reply
Sign up and join the conversation on Discord