Find answers from the community

s
F
Y
a
P
Updated 2 years ago

Wondering if there s any way to save

Wondering if there's any way to save/load an index to S3 or some other filestore? I suppose we'd then need to download the whole index every time a query is run?
j
b
11 comments
this is a good idea. currently most indices are serializable to a json file on disk, and when loaded they're stored in memory
some indices like weaviate/pinecone are backed by respective db's
right - i didn't want to mess with weviate for a json index, so I'm saving to disk
but my app is on fly.io and working with storage is tricky
it'd be better to be able to just store the indexes in s3
or what about in postgres?
yep makes sense
that would be super eary
i want to add "serializer" classes so that you can save this stuff to external stores
yeah, makes sense
Add a reply
Sign up and join the conversation on Discord