Find answers from the community

Updated 3 months ago

Node parers

Hi all i am using simplenodeparser to split document to nodes and the document is split into 6 nodes. Is there any way the split the document into much smaller nodes

Plain Text

documents = SimpleDirectoryReader('/home/ubuntu/llama_index/examples/paul_graham_essay/data').load_data()

parser = SimpleNodeParser()

nodes = parser.get_nodes_from_documents(documents)
index = GPTSimpleKeywordTableIndex(nodes, service_context=service_context)

3 comments

LLogan M

You can!

You just have to pass a text splitter into the node parser, which will have an option for the chunk size and chunk overlap

jjagadeeshj

Thanks @Logan M i got it working. How do i enable streamingResponse for my query ? BTW i'm using a llama LLM

LLogan M

Hmmm I'm not sure if streaming is supported for custom LLMs 😅 would need some work in the codebase to support I think

Add a reply