Find answers from the community

Updated 3 months ago

Now that the ServiceContext is

Now that the ServiceContext is deprecated in favor of Settings, I want to test the performance of different chunk sizes with multiple VectorStoreIndex, where do I have to specify now the chunk_size for this to work?

3 comments

LLogan M

You can specifuy the chunk size as a global

Plain Text

Settings.chunk_size = 512

Or specify the node parser/text splitter and pass that into the index

Plain Text

from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(chunk_size=512)
index = VectorStoreIndex.from_documents(..., transformations=[splitter])

LLogan M

You'll notice that the index takes a transformations array, similar to the ingestion pipeline

ssgaseretto

Thanks a lot!

Add a reply