Find answers from the community

Updated 3 months ago

Object index

Does anyone know how to persist an ObjectIndex? The actual index is internal (_index) and when persisting it it does not seem straight forward to reload it from disk. I'm curious why this isn't supported like it is with other types of indexes.

4 comments

LLogan M

You can definitely persist the underlying internal index, but that doesn't save the node to object mapping

And that node to object mapping is why there hasn't been a better way to do this yet 😅 but it's on the todo list somewhere

MMillianz

Thanks, that's the exception I ran into when I persisted the internal structure (invalid object mapping). I'm new to LlamaIndex and don't feel super comfortable yet, but as I onboard more maybe I can try and contribute this.

LLogan M

I think the solution might be just pickling the object mapping, along with persisting the underlying index as normal

But at the same time, relying on pickles kind of sucks haha

MMillianz

Hah yeah, I'll give it a try though. A sucky solution might be better than none - at least while developing locally and not wanting to wait for the indexing every time.

Add a reply