Find answers from the community

Updated 3 months ago

Node parers

Hi all i am using simplenodeparser to split document to nodes and the document is split into 6 nodes. Is there any way the split the document into much smaller nodes
Plain Text
documents = SimpleDirectoryReader('/home/ubuntu/llama_index/examples/paul_graham_essay/data').load_data()

parser = SimpleNodeParser()

nodes = parser.get_nodes_from_documents(documents)
index = GPTSimpleKeywordTableIndex(nodes, service_context=service_context)
L
j
3 comments
You can!

You just have to pass a text splitter into the node parser, which will have an option for the chunk size and chunk overlap
Thanks @Logan M i got it working. How do i enable streamingResponse for my query ? BTW i'm using a llama LLM
Hmmm I'm not sure if streaming is supported for custom LLMs πŸ˜… would need some work in the codebase to support I think
Add a reply
Sign up and join the conversation on Discord