The community member is using load_index_from_storage and index.storage_context.persist(...) to dynamically load and save indexes in their API. They have created their own StorageService class to handle file/folder read/write operations, allowing them to switch from local storage to services like S3. However, they are unsure how to override the default behavior of persist and load_index_from_storage to use their custom StorageService class.
In the comments, another community member suggests that the issue can be resolved by passing an fsspec object to change the storage backend to S3 or Google Cloud Buckets. They also provide a link to the relevant documentation on using a remote backend.
I'm using load_index_from_storage and index.storage_context.persist(...) to dynamically load and save the indexes as they're created within my API. I already do my own preprocessing steps on the document and have created my own set of StorageService class that handles read/write file, folder etc so I can easily switch from localstorage (local dev) to e.g. S3 or something else - but not sure how this bit can work with that dependency injection approach - @Logan M any pointers 😄
