hey everyone how are you do you guys

I have used pinecone and GPTSimpleVectorIndex and with limited testing, had the same results for both. So now I mostly use simple vector index for testing, there are definitely use cases where using SaaS has advantages, but unless I specifically need the SaaS, I like simple vector index.

Can I persist the data using GPTSimpleVectorIndex?

Yes, you can save and load it to disk (or string, if you need to put it on s3 or something)

ok, I will give it a try, but maybe I still need a database for the scale of the data I will store

Hey @Logan M , could you please elaborate on how to persist an index to AWS S3? TIA!

Check out this page in the docs
https://gpt-index.readthedocs.io/en/latest/how_to/storage/save_load.html#using-a-remote-backend

Great, thanks!

Hey @Logan M , I tried using this guide to save my index in the S3 but apparently there's a bug in there.
Specifically, the problematic lines are these:
https://github.com/jerryjliu/llama_index/blob/main/llama_index/storage/storage_context.py#LL87C11-L87C11

As I'm using a Windows laptop, Path changes my S3 path that has slashes in it, and replaces them with backslashes on which I get an error as S3 path should only use slashes.

Hmmm that's a super tricky bug, good catch! Maybe open a PR with this and see if it's a good enough solution (if fs is provided, I think it makes sense to let the user have total control of the path)

Yeah, that's what I thought

Will open s PR