Find answers from the community

Updated 2 months ago

Hi I am having problem using auto-merge

Hi I am having problem using auto-merge with llama index.
Question
I am trying to test the auto merge with llama_index in my RAG system. For this i have created couple of python files which gets input from command lines. I run these python scripts from my nodejs application.
Here i have two files

  • doc-loader-llama-index.py
  • retriever-merged-chunks-llama-index.py
first file loads the pdf's text create nodes, add nodes in to doc store(mongodb), child nodes into vector store (pinecone). I added multiple documents by calling this file each times. I have around 43 documents added. I can see i have child nodes added into pinecone. I can also see 4 collections being created in mongodb ( 'docstore/data', 'docstore/metadata', 'docstore/ref_doc_info', 'index_store/data')

second file is to retreive the chunks/nodes with auto merge. This is where I am getting a error. The error message is as below:

---
Error: Command failed: python3 /home/fm-pc-lt-214/proj/qdabra/doc-gpt/py-libs/retriever-merged-chunks-llama-index.py "MY OPENAI APIKEY" "MY PINECONE APIKEY" "MY PINECONE INDEX" "MY PINECONE ENVIRONMENT" "What references are cited in the Creating True Copies Standard Operating Procedure?" "5"
Traceback (most recent call last):
File "/home/fm-pc-lt-214/proj/qdabra/doc-gpt/py-libs/retriever-merged-chunks-llama-index.py", line 71, in
autoMergeIndex = load_index_from_storage(
File "/home/fm-pc-lt-214/proj/qdabra/doc-gpt/py-libs/venv/lib/python3.8/site-packages/llama_index/indices/loading.py", line 40, in load_index_from_storage
raise ValueError(
ValueError: Expected to load a single index, but got 43 instead. Please specify index_id.
----
PS. I have replaced acutal apikeys with text here.

I am not sure why i am getting this error.
Is it not possible to add multiple documents and make query with automerge ?
L
s
4 comments
It seems like you made 43 indexes? Can you share some code for how you did this?
i have 2 files. one is to ingest data which extracts texts from pdf, creates nodes and add them in document store and vector store. I am using mongodb and pinecone.
other file is used retreive related chunks/nodes to user query.
ill need to try it myself when I get a chance, no idea whats happening lol
Add a reply
Sign up and join the conversation on Discord