Find answers from the community

Updated 3 months ago

MongoDB Atlas

Using MongoDB Atlas, is there a way to load only the index without loading any document (since there are not in the database) ?
https://gpt-index.readthedocs.io/en/latest/examples/vector_stores/MongoDBAtlasVectorSearch.html this page only uses document loader
V
L
82 comments
I think i might have misunderstood what is mongodb created for
My use case is that i want to stores my indexes, indexed in a vector store format with GPTVectorStoreIndex, and query over them without having to load any documents or index any documents at first.
Just using .json with an embedded documents in it
Should I be able to do that with MongoDB Atlas ? If yes, how ?
And if not, is my tool to do that would be a vector database such as Pinecone ?
You should be able to do it with at atlas or pinecone πŸ‘€

Just need to setup the vector store and storage context and create the index

Then to "load" the index, setup the vector store again and do something like

index = VectorStoreIndex.from_vector_store(vector_store)
I guess that's what I'm doing here:
Attachment
image.png
But then I get this error: File "c:\Projets\IA Chat Local\Sources\AzureOpenAI\app copy.py", line 203, in get_json index = VectorStoreIndex.from_vector_store(vector_store="DTU") File "C:\Users\sxd-i\AppData\Local\Programs\Python\Python310\lib\site-packages\llama_index\indices\vector_store\base.py", line 60, in from_vector_store if not vector_store.stores_text: AttributeError: 'str' object has no attribute 'stores_text'
And I instanciated my db like that:
Attachment
image.png
I think I might have errors on: 1- the format of the index in Mongo
2 - The calling part of the index
Ah, looks like a small bug, the vector store is missing that attribute 😒
But still, is the index instanciated in the right way in Mongo ?

I can't find/understand how llama_index processes it when using it for the query so idk wich part he's taking
oh wait, I read this closer, you had an error in your code, one sec!
index = VectorStoreIndex.from_vector_store(store) <- should be that instead of passing in a string
yeah i had this result before, it fixed it !
Where i have the problem now
is that as a result, i should have an entire json with some response
I guess that when doing that, it doesnt take any info from the index, and thus can't query over it
πŸ€” it should work... one sec. I need to test this part of the code for another PR, so may as well test it for this too lol
I should have something like that
Attachment
image.png
I can give you my code if you want
nah its all good. Just figuring out how to create a mongodb instance lol
Does it work if you query right after creating the initial index? I can't even get an initial response 😦
I can see it inserted everything though
I don't even create it, i import it trough my terminal
like the files
that's why i'm not sure of the format
mongoimport --uri mongodb+srv://xxxxxxx:xxxxxx@indexatlasdb.vwrbmy3.mongodb.net/DTU --collection index_store --type json --file index_store.json
ohhhhhhh you uploaded the entire index into mongodb? I thought you just used the vector store πŸ˜…

Using just the vector store takes care of everything, no need to worry about index_store or docstore
I'm on a slightly different unreleased version of llama-index, so it will look slightly different for you, but every entry in the vector store includes the node data
Attachment
image.png
I tried to do it this way but i didnt managed to sooooo
whent for the hard one x)
Would mind to show me the part of the code doing it ?
But also, I'm struggling to get an answer for a query lol from both the initial index and the loaded index
trying to figure out why the mongo query returns none 😦
But also, this is my first time looking at mongo db code hahaha
Hmm I think I missed a step, I think I need to configure a mapping/embedding field in my db first
it doesnt even create the index for me :/// It created it one time, then I deleted it to be sure
and now nothing
and still get the same error as before
Ok, when querying like this:
Attachment
image.png
I have an output as Null:
Attachment
image.png
because i have no index at all or i "load" nothing
and with this:
Attachment
image.png
No index is created, and i get a program error
even though it created an index one time on the first time
aaaaaah that's pisses me off, i change nothing and it doesnt want to be created anymore
I got it to work!
ok, so the screenshot I shared before is correct (for the creation and loading)
but in your mongodb portal, you need to setup a search index. Use the JSOn editor and do something like this
Attachment
image.png
wait a few minutes, and then queries start working
this is not well documented at all LOL
I got the setup process from the langchain docs actually. But even those instructions were bad https://python.langchain.com/docs/modules/data_connection/vectorstores/integrations/mongodb_atlas_vector_search
let me know if you have troubles creating the search index. Once I did that in their UI, everything worked πŸ™‚
I will try it tomorrow and I let you know !
from_documents for the indexation
from_vector_storefor the retrieval
long time no see this error
Attachment
image.png
i had it a way ago
then it disappeared idk how
and it came back
with this technique
Ok ! I got it all working !!!
few updates for your knowledge:

1 - If you want to add a document to an existing index, you don't need to recreate the index from 0 with all the base documents. You can just add the new document to the end of the existing index
2 - when you create the serach index, do not change the name "default" here either it'll not work
Attachment
image.png
3 - you can query by two different manners:
3.1 - if you want to query over an existing index named db_name for exemple or dont add it if you want to use the default one
3.2 - this way, while adding new documents over the existing index, you can query too with the new updated index
Attachment
image.png
and the index is created also with this technique (if not created before) with the db_name argument of your choice
Et voilΓ  !
Btw you can have also different indexes:
Attachment
image.png
and the search engines linked to every one of them:
Attachment
image.png
Ayyyy thanks for the detailed stuff! Amazing! πŸ‘ πŸ‘
Add a reply
Sign up and join the conversation on Discord