Has anyone else noticed HuggingFaceLLMs just hanging on their machine once you get to query? I have been unable to get a response from them and ultimately have to do a KeyboardInterrupt

Hey All! I implemented a Custom Embedding class to support the PaLM API, and have run into an error once I get to query_engine.query(). The embedding variable passed down from the base query engine is empty, so I get a numpy error

What are some best practices for persisting indexes? I am using ChromaDB and adding documents/embeddings in a separate process (outside of Llama Index). I am interested in building composable indexes that are groups of keywords that relate to particular documents. As of right now, I have lots of separate documents (200k+) and i don't get super accurate results. My plan has been to separate these documents into different categories that have metadata associated with them so that there can be a more accurate retrieval process. With that structure, how do we store these indexes?

Where does llama_index extract the query it sends to a vector store? I am trying to alter the query it sends to customize the chatbot behavior

Is there any way to control how the LLM uses the source_nodes? I sometimes find that source_nodes are accurate, but the response generated hallucinates names or properties. Can we force the engine to look for particular properties that we know exist in all documents?

Is there a way to customize nodes and the data associated with them to help the refine process look for particular fields? I guess I want my chatbot to always return a particular property found on each node and am unsure as to how to get it to do that

How do we use a custom response_synthesizer in a chat_engine? I'd like to use structured_answer_filtering with my chat engine

I'm noticing the chatprompts dont have much effect on the output from OpenAI. Where can we see what the string looks like when its being sent to OpenAI for completion? I guess I just want to understand where the the prompts come into play in the pipeline

When working with chromadb directly, and loading the index frm VectorStoreIndex.from_vector_store(), I get the following error when using the chat_repl()

chromadb.errors.InvalidDimensionException: Embedding dimension 768 does not match collection dimensionality 384

I am using OpenAI as the LLM, im assuming this is because when i do chroma_collection.upsert() (via there API) that this uses their default embedding model which doesn't match the the dimensions that OpenAI expects?

Might be a dumb question, but not entirely sure:

Does using a vector_store persist the index automatically via the StorageContext? In my case, I'm using ChromaDB. Do I need to persist indexes to disk instead?

How do we add a custom prompt to a chat_engine with OpenAI. The examples I found here in the Discord seem to use langchain and are potentially out-of-date (still referencing gpt-index as the package name). What's the right way to do this? Have looked here: https://gpt-index.readthedocs.io/en/stable/core_modules/query_modules/chat_engines/usage_pattern.html