Find answers from the community

Updated 2 years ago

Chunk size

At a glance

The community member encountered a ValueError related to the chunk overlap being larger than the chunk size in the llama_index library. The community member asked how to normally solve this issue, suggesting to reindex again. In the comments, another community member suggested that reindexing might help, but also mentioned that it depends on the setup settings.

aautratec

as i saw followign message. normally how to solve it ? reindex again ?

9 frames
/usr/local/lib/python3.10/dist-packages/llama_index/langchain_helpers/text_splitter.py in init(self, separator, chunk_size, chunk_overlap, tokenizer, backup_separators, callback_manager)
38 """Initialize with parameters."""
39 if chunk_overlap > chunk_size:
---> 40 raise ValueError(
41 f"Got a larger chunk overlap ({chunk_overlap}) than chunk size "
42 f"({chunk_size}), should be smaller."

ValueError: Got a larger chunk overlap (-4) than chunk size (-45), should be smaller.

1 comment

LLogan M

Maybe reindex? But it also depends on some of your setup settings

Add a reply