Find answers from the community

Updated 3 months ago

I am using unstructured loader to load

I am using unstructured loader to load documents in my chroma vector store. Now before i run vectorstorefromdocuemnts to load them in to my db, can i get the nodes in a variable? I want them in a variable so that i can setup a recursive retriever for questions+ summaries

Plain Text

dir_reader = SimpleDirectoryReader('./data/download', file_extractor={
  ".html": UnstructuredReader(),
})
documents = dir_reader.load_data()

2 comments

rrahul

how are you defining the base node and vector store? what type is it?

FFried cheese

I am not having a variable called base_nodes when I am directly inserting in my db. I tried getting base_nodes from my document variable, as in:

Plain Text

node_parser = SentenceSplitter(chunk_size=512)
base_nodes = node_parser.get_nodes_from_documents(documents)

Add a reply