Hello, how do I convert a list of TextNode objects into Document objects? The reason I'm asking is because I'm try to run the Zephyr 7B Beta Collab provided, but it doesn't work when I start with a list of TextNode objects instead of a Document object. The error I get is
'TextNode' object has no attribute 'get_doc_id'
My code is:
document: Document = Document(text=txt_str, metadata=metadata or {}) # same approach as SimpleDirectoryReader
documents: list[Document] = [document]
node_parser = SentenceWindowNodeParser.from_defaults(
window_size=3,
window_metadata_key="window",
original_text_metadata_key="original_text",
)
nodes: list[TextNode] = node_parser.get_nodes_from_documents(documents) # this returns TextNodes rather than a Document
llm = HuggingFaceLLM(
model_name="HuggingFaceH4/zephyr-7b-beta",
tokenizer_name="HuggingFaceH4/zephyr-7b-beta",
query_wrapper_prompt=PromptTemplate("<|system|>\n</s>\n<|user|>\n{query_str}</s>\n<|assistant|>\n"),
context_window=3900,
max_new_tokens=256,
# model_kwargs={"quantization_config": quantization_config},
# tokenizer_kwargs={},
generate_kwargs={"temperature": 0.7, "top_k": 50, "top_p": 0.95},
messages_to_prompt=messages_to_prompt,
device_map="cpu",
)
service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model)
vector_index = VectorStoreIndex.from_documents(nodes, service_context=service_context) # Error happens here