LlamaIndex
Log in

Find answers from the community

Home
Members
Murk418
M
Murk418
Offline, last seen 3 months ago
Joined September 25, 2024
MMurk418
·

Storage

Hey! Would you happen to know if its possible to load a VectorStoreIndex by passing the docstore, graph_store, index_store, and vector_store directly? I'm currently downloading the files from a GCS bucket, which means that by default I would need to download them to a local directory, before loading the index from the persist_dir. But I was wondering if it would be possible to instead just pass the docstore, graph_store, index_store, and vector_store directly after downloading them into memory from the storage bucket? Any insights would be much appreciated. Thanks again!
2 comments
M
L
MMurk418
·

Progress

Hey, when building a VectorStoreIndex, I know you can add the show_progress=True param to show the progress in the terminal like this:

Plain Text
index = VectorStoreIndex.from_documents(docs, show_progress=True)


But I was wondering if its possible to access the progress programmatically through a callback manager or some other means other than just through what gets logged to the console? I couldn't find anything in the docs, and from what I can tell from the source code, that boolean show_progress param doesn't result in any callback that can be accessed without modifying the source. But idk. Any help would be very appreciated, thanks!
2 comments
M
L
MMurk418
·

Does anyone know if there is a way to

Does anyone know if there is a way to print the result of a custom prompt template? The following is my current implementation, and I'd like to be able to view the actual resulting prompt that gets sent to the LLM. Thanks in advance!:

Plain Text
query_engine = index.as_query_engine(text_qa_template=Prompt(CHAT_PROMPT), verbose=True)
        chat_engine = index.as_chat_engine(
            query_engine=query_engine,
            chat_mode="context",
            chat_history=chat_history,
            query_str=query,
            similarity_top_k=2,
            verbose=True,
        )
7 comments
L
M
MMurk418
·

Terraform Registry

@Marcos Duarte @WhiteFang_Jr @praveenks30#120294 @ash7 @OverclockedClock @BP

Hey, just leaving this here for some of ya'll I saw were interested in MongoDB Atlas incase you're still interested, or for anyone else that comes across this issue. From what I can tell @Logan M , you are correct that Atlas doesn't currently support the ability to create a search index via client libraries/api/mongosh. At least not for the M0 free cluster, or the other M2/M5 shared clusters (reference). Although it looks like maybe its possible for the dedicated clusters, but I haven't tried it yet.

However, I've been able to find one weird solution that seems to work. You can use the mongodbatlas Terraform provider to provision a mongodbatlas_search_index resource. The only issue with this (other than it being a pain in the ass if you aren't already using Terraform lol) is that you can't create a collection with Terraform, and although provisioning a search index without an existing collection will return a 200, it will fail to actually create the index. Meaning you would need to run some kind of external script in order to create an empty collection before provisioning the index.

Here's what I came up with --------->
7 comments
M
L
B