Refreshing an Existing Vector Store

At a glance

The community member has a question about refreshing an existing vector store. They note that the normal VectorStoreIndex allows passing a show_progress flag to display a progress bar, but they don't see such an option for the refresh_index method. They would like to know if there is a way to show progress when refreshing the index, and see information about the chunks being modified or refreshed.

In the comments, another community member acknowledges that there is no progress bar at the moment, but suggests it should be easy to add. When asked how to add it, the community member proposes that in the refresh_ref_docs method, they could use tqdm and possibly pass a show_progress bool.

There is no explicitly marked answer in the provided information.

SSaltuk

One question regarding refreshing an existing vector store.
As far as i can see the normal VectorStoreIndex allows to pass in the show_progress flag which shows the nice tqdm-like bar when generating the chunks.
I don't see such an option for the refresh_indexmethod. Is there a way to show progress there? Would especially be interesting to see how many chunks/what kinds of chunks are being modified/refreshed.
Currently i'm executing it like this and unfortunately i don't see any indication.

Plain Text

storage_context = StorageContext.from_defaults(persist_dir=storage_dir)
index: VectorStoreIndex = load_index_from_storage(storage_context=storage_context)
index.refresh_ref_docs(documents)

3 comments

LLogan M

hmmm yea no progress bar at the moment. Should be easy enough to add

SSaltuk

How exactly? Would be cool if i can add it temporarily by just overwriting some files

LLogan M

In the refresh_ref_docs method, would just need to use tqdm and maybe also pass in a show_progress bool

Attachment

Add a reply

Find answers from the community

Refreshing an Existing Vector Store