The community member is having difficulty understanding the default chunking and tokenization performed by VectorStoreIndex.from_documents(). Another community member suggests using the ingestion pipeline, which is more transparent about the process. The ingestion pipeline documentation is provided as a reference.
It isn't clear to me the default chunking and tokenization that is being performed under VectorStoreIndex.from_documents(). Usually I can figure this out on my own, but having difficulty. Is this documented somewhere?
