Find answers from the community

Updated 2 months ago

how do I increase the chunk size in simply directory reader?

how do I increase the chunk size in simply directory reader?
k
2 comments
Please stand by @Callam as I search the knowledge base. I normally take about 20 seconds or so. :wave:
----------------------
To increase the chunk size in LlamaIndex, you need to adjust the chunk_size parameter in the Settings class from the llama_index.core module. Here is an example of how to do it:

Plain Text
from llama_index.core import Settings

# initialize settings (set chunk size)
Settings.chunk_size = 1024  # You can set this to any value you want


This setting will be used when documents are ingested into an index, where they are split into chunks of the specified size.

Please note that changing the chunk size will change the embeddings that are calculated. A smaller chunk size means the embeddings are more precise, while a larger chunk size means that the embeddings may be more general, but can miss fine-grained details.

For more information, you can refer to the LlamaIndex documentation () and this guide on optimizing basic strategies ().
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
Add a reply
Sign up and join the conversation on Discord