Find answers from the community

Updated 6 months ago

Hey guys, I created an index for each

Hey guys, I created an index for each document and saved each in a directory, how do I load them into one index or engine?
W
a
5 comments
Is there a reason to have index for each document?

Also you cannot have single index out of all the indexes directly. You will have to use maybe FusionRetriever, HybridRetriever to combine retriever from all the indexes to prepare one index on top of it.
Well I needed to do every document alone because doing them altogether gave an error due to one of the documents and now I am out of LlamaParse daily limit. Would using the retreiver take more time?
You can iterate over one file at a time and add the returned doc into the final data. Once all the files are done you can build the final index.

If you wanna parse separately. Store the parsed docs in a file.
Excuse my ignorance but isn't it sometimes better to have an index for every document? How is it later combined? I guess an agent is needed?
I will be doing one index now & parse all documents altogether that I know which document was causing the issues but I still would like to know
Add a reply
Sign up and join the conversation on Discord