Find answers from the community

Updated 2 years ago

Hey a quick question on persistent

At a glance

The community member has a question about persistent storage for indices and graphs they have generated. They have tried using S3 storage but encountered issues with serializing and deserializing. The comments suggest checking the StorageContext and provide a link to documentation on saving and loading data. Another community member mentions that the project currently supports saving directly to MongoDB and that adding S3 support should be easy. They are open to contributions and a community member has submitted a pull request for initial S3 support. There is no explicitly marked answer, but the community members are discussing potential solutions.

Useful resources
Hey, a quick question on persistent storage. To store the indices and graphs that are generated, should I be using a database or an S3 storage? I have tried using S3, but I am getting errors with serialising and deserialising.

Thanks in advance for any tips! πŸ™
c
d
d
8 comments
maybe check StorageContext
@chanansh thanks for helping answer
@pineapple right now we support directly saving into MongoDB, should be very easy to add S3
will get to it in the near future, also open to contribution
I am new here. Happy to contribute. Still figuring why parsing one document takes so long.
how long is your document and how long is it taking?
Add a reply
Sign up and join the conversation on Discord