Find answers from the community

Home
Members
bin4ry_d3struct0r
b
bin4ry_d3struct0r
Offline, last seen 2 months ago
Joined September 25, 2024
This may not be related to LlamaIndex, but does anyone know how to directly access the context window? I do not mean setting the window size via Settings.context_window but accessing the actual contents inside the window, which permanently stores the system prompt and the query-response exchanges up to the token limit.

I want to give the user the option to 1) change the system prompt on the fly depending on model responses; and/or 2) manually edit the model responses more to the user's liking. I figured the most convenient way to do this is to directly access the context window contents.

If there's no way to do this, then I guess the only recourse is via the chat store or memory buffer, but the system prompt is not stored in either.
21 comments
b
L
Is there a reason why similarity_top_k can be set directly when initializing a VectorStoreIndex as a query engine but not as a chat engine?
6 comments
L
b
My basic understanding of the VectorStoreIndex is apparently somewhat confused. The class file vector_store/base.py describes it as:

An index that is built on top of an existing vector store.

Key words here being an existing vector store. As such, I thought a vector store was built when the index is initialized, such as with:

Plain Text
index = VectorStoreIndex(nodes)
index.storage_context.persist(persist_dir = persist_dir)


However, when I try to access the underlying vector store on top of which the index was supposed to be built:

Plain Text
storage_context = StorageContext.from_defaults(persist_dir = persist_dir)
index = load_index_from_storage(storage_context = storage_context)
vector_store = index.from_vector_store()


... it tells me that I need to provide a vector store, and therein lies my confusion. I'm trying to access the vector store from the index which I thought was located in persist_dir when VectorStoreIndex() was initialized, not build a new one.

Where is the underlying vector store for the index and how can I access it, please? (For context, I want to retrieve the vector store from the index in order to pass it as an argument to VectorMemory.)
8 comments
L
b
U
Without seeing a specific error output, it's difficult to answer your question.
6 comments
M
b
Hi, I'm having trouble tracing the cause of this issue in the source code (v0.9.14.post3).

I built a hybrid retriever:

Plain Text
vector_retriever = index.as_retriever(similarity_top_k = 3)
bm25_retriever = BM25Retriever.from_defaults(docstore = index.docstore, similarity_top_k = 3)

retriever = QueryFusionRetriever(
    llm              = llama,
    mode             = "reciprocal_rerank",
    num_queries      = 1,  # set this to 1 to disable query generation
    retrievers       = [bm25_retriever, vector_retriever],
    similarity_top_k = 3,
    use_async        = True,
    verbose          = True
)

query_engine = RetrieverQueryEngine.from_args(retriever)


Note that the llm argument is initialized to be llama which is defined elsewhere as one of the Llama-2 chat models. Yet, when I call _queryengine.query(), it still sends API calls to OpenAI for completion. I thought it shouldn't be doing that because I've passed in a non-default model into the llm argument in the QueryFusionRetriever class.

What gives?
10 comments
L
b
So, if I want to build a bot that combines semantic search with lexical search (using QueryFusionRetriever), there's no way to incorporate chat history?
12 comments
b
L
I'm wondering if this is a bug.
6 comments
L
b
Otherwise, what's the point?
11 comments
a
b
I've had to reduce Settings.chunk_size by half as well as similarity_top_k in the query engine to get working what worked fairly well in v.0.9.
22 comments
b
L
Yay ... all my old code is now broken 🥳
11 comments
b
L
I would find it very hard to believe that I'm the first one to think of such a basic idea, but I'm looking for examples of open source projects and haven't come across any.
8 comments
L
b
@kapa.ai The nodes in my VectorStoreIndex have sequential identifiers. How do I add new nodes to the index with identifiers starting from the identifier of the last node in the index?
14 comments
k
b
Okay, nvm ... apparently, there is someone in the RAG space named Jason Liu who does actually own a Porsche, so it wasn't a shot against Jerry with his name misspelled.
1 comment
j
Very strange!
6 comments
L
b
Noob question: What is the practical difference between working with documents vs. working with nodes?

For example:

index = VectorStoreIndex.from_documents(documents, service_context = service_context)

vs.

index = VectorStoreIndex(nodes, service_context = service_context)

Are there significant (dis)advantages to each?
1 comment
L