Find answers from the community

Updated 2 months ago

Storage

At a glance

The community member is experiencing a significant slowdown when retrieving a storage connection from disk. The comments suggest several solutions:

- Don't save to disk, as it's not meant for large datasets

- Use storage integrations instead

- Pickle the entire storage context, which can be faster

- Store everything in a Chroma DB store, as suggested by another community member

The comments indicate that the Chroma DB store solution worked for the community member who was experiencing the slowdown.

I’m seeing a significant slow down when I retrieve a storage connect from disk - is this there simmering I can do to figure out what is doing in here?
L
c
5 comments
Don't save to disk? It's really not meant for large datasets
There's storage integrations for everything
Ive also seen people just pickle the entire storage context too, which is fast
@Logan M you saved me again - I had to put everything in chroma db store
The docs should push users to do this as much as possible when going through QuickStarts
Add a reply
Sign up and join the conversation on Discord