TextNode to an IndexNode in LlamaIndex, you can use the from_text_node class method provided by the IndexNode class. This method takes a TextNode and an index_id as arguments and returns an IndexNode.from llama_index.core.schema import TextNode, IndexNode # Create a TextNode text_node = TextNode(text="<text_chunk>", id_="<node_id>") # Convert TextNode to IndexNode index_node = IndexNode.from_text_node(node=text_node, index_id="<index_id>")
<text_chunk> with the text content you want to store in the node, <node_id> with the unique identifier for the node, and <index_id> with the unique identifier for the index.TextNode instances that you want to convert to IndexNode instances, you can simply use a loop to process each TextNode. Here's an example:from llama_index.core.schema import TextNode, IndexNode
# Assume text_nodes is a list of TextNode instances
text_nodes = [TextNode(text=f"<text_chunk_{i}>", id_=f"<node_id_{i}>") for i in range(10)]
# Convert each TextNode to an IndexNode
index_nodes = [IndexNode.from_text_node(node=text_node, index_id=f"<index_id_{i}>") for i, text_node in enumerate(text_nodes)]<text_chunk_{i}>, <node_id_{i}>, and <index_id_{i}> with the text content, node identifier, and index identifier for each node, respectively.VectorStoreIndex with IndexNode instances, you can pass the list of IndexNode instances to the VectorStoreIndex constructor. Here's an example:from llama_index.core import VectorStoreIndex # Assume index_nodes is a list of IndexNode instances index = VectorStoreIndex(index_nodes)
index_nodes with your list of IndexNode instances.StorageContext to the VectorStoreIndex constructor, especially if you're using a custom vector store. The StorageContext can be created using the StorageContext.from_defaults method and the custom vector store as an argument.from llama_index.core import VectorStoreIndex, StorageContext from llama_index.vector_store import YourCustomVectorStore # Assume index_nodes is a list of IndexNode instances # Assume your_vector_store is an instance of YourCustomVectorStore storage_context = StorageContext.from_defaults(vector_store=your_vector_store) index = VectorStoreIndex(index_nodes, storage_context=storage_context)
YourCustomVectorStore with the class of your custom vector store, and your_vector_store with an instance of your custom vector store.SentenceSplitter class to split your base nodes into smaller chunks, and then create IndexNode instances from these chunks. These IndexNode instances will have the same node_id as the base node, effectively creating a reference to the bigger chunk.from llama_index.core.node_parser import SentenceSplitter
from llama_index.core.schema import IndexNode
# Define the sizes for your sub-chunks
sub_chunk_sizes = [128, 256, 512]
# Create SentenceSplitter instances for each sub-chunk size
sub_node_parsers = [SentenceSplitter(chunk_size=c, chunk_overlap=20) for c in sub_chunk_sizes]
# Assume base_nodes is a list of your base nodes
all_nodes = []
for base_node in base_nodes:
for n in sub_node_parsers:
# Split the base node into sub-nodes
sub_nodes = n.get_nodes_from_documents([base_node])
# Create IndexNode instances from the sub-nodes
sub_inodes = [IndexNode.from_text_node(sn, base_node.node_id) for sn in sub_nodes]
# Add the IndexNode instances to your list
all_nodes.extend(sub_inodes)
# Also add the original base node to your list
original_node = IndexNode.from_text_node(base_node, base_node.node_id)
all_nodes.append(original_node)base_nodes with your list of base nodes.RecursiveRetriever. The RecursiveRetriever will retrieve the smaller chunks and then follow the references to retrieve the bigger chunks, providing more context for synthesis.