Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
π
π
π
Powered by
Hall
Inactive
Updated 3 months ago
0
Follow
How can I get llama index to use the
How can I get llama index to use the
Inactive
0
Follow
d
damon
2 years ago
Β·
How can I get llama_index to use the entire document i pass in as context
L
d
5 comments
Share
Open in Discord
L
Logan M
2 years ago
If the document is too long, it will be broken into chunks
You can try increasing the chunk_size_limit though (the default is 1024)
service_context = ServiceContext(..., chunk_size_limit=200)
L
Logan M
2 years ago
I think this is what you are getting at?
d
damon
2 years ago
when prompting i mean
d
damon
2 years ago
for one prompt i want it to use the entire context
L
Logan M
2 years ago
Hmm maybe a longer explanation will help us get on the same page
When you insert documents, they are broken into nodes/chunks, as I detailed above
When using a vector index, each node/chunk is embedded
At query time, the query is also embedded, and the top k chunks are retrieved from the index (by default, the top k is two)
Using the default response mode (compact), all the text from the retrieved nodes, along with the query, is sent to the LLM
If the retrieved text is too long for the LLM to process, then it's broken into pieces and an answer is refined across multiple LLM calls
What's the exact issue you are running into then?
Add a reply
Sign up and join the conversation on Discord
Join on Discord