bin4ry_d3struct0r

Context window

This may not be related to LlamaIndex, but does anyone know how to directly access the context window? I do not mean setting the window size via Settings.context_window but accessing the actual contents inside the window, which permanently stores the system prompt and the query-response exchanges up to the token limit.

I want to give the user the option to 1) change the system prompt on the fly depending on model responses; and/or 2) manually edit the model responses more to the user's liking. I figured the most convenient way to do this is to directly access the context window contents.

If there's no way to do this, then I guess the only recourse is via the chat store or memory buffer, but the system prompt is not stored in either.

21 comments

bbin4ry_d3struct0r

Why similarity_top_k can be set directly when initializing a VectorStoreIndex as a query engine but not as a chat engine

Is there a reason why similarity_top_k can be set directly when initializing a VectorStoreIndex as a query engine but not as a chat engine?

6 comments

bbin4ry_d3struct0r

My basic understanding of the

My basic understanding of the VectorStoreIndex is apparently somewhat confused. The class file vector_store/base.py describes it as:

An index that is built on top of an existing vector store.

Key words here being an existing vector store. As such, I thought a vector store was built when the index is initialized, such as with:

Plain Text

index = VectorStoreIndex(nodes)
index.storage_context.persist(persist_dir = persist_dir)

However, when I try to access the underlying vector store on top of which the index was supposed to be built:

Plain Text

storage_context = StorageContext.from_defaults(persist_dir = persist_dir)
index = load_index_from_storage(storage_context = storage_context)
vector_store = index.from_vector_store()

... it tells me that I need to provide a vector store, and therein lies my confusion. I'm trying to access the vector store from the index which I thought was located in persist_dir when VectorStoreIndex() was initialized, not build a new one.

Where is the underlying vector store for the index and how can I access it, please? (For context, I want to retrieve the vector store from the index in order to pass it as an argument to VectorMemory.)

8 comments

bbin4ry_d3struct0r

Without seeing a specific error output,

Without seeing a specific error output, it's difficult to answer your question.

6 comments

bbin4ry_d3struct0r

Query engine

Hi, I'm having trouble tracing the cause of this issue in the source code (v0.9.14.post3).

I built a hybrid retriever:

Plain Text

vector_retriever = index.as_retriever(similarity_top_k = 3)
bm25_retriever = BM25Retriever.from_defaults(docstore = index.docstore, similarity_top_k = 3)

retriever = QueryFusionRetriever(
    llm              = llama,
    mode             = "reciprocal_rerank",
    num_queries      = 1,  # set this to 1 to disable query generation
    retrievers       = [bm25_retriever, vector_retriever],
    similarity_top_k = 3,
    use_async        = True,
    verbose          = True
)

query_engine = RetrieverQueryEngine.from_args(retriever)

Note that the llm argument is initialized to be llama which is defined elsewhere as one of the Llama-2 chat models. Yet, when I call _queryengine.query(), it still sends API calls to OpenAI for completion. I thought it shouldn't be doing that because I've passed in a non-default model into the llm argument in the QueryFusionRetriever class.

What gives?

10 comments

bbin4ry_d3struct0r

So, if I want to build a bot that

So, if I want to build a bot that combines semantic search with lexical search (using QueryFusionRetriever), there's no way to incorporate chat history?

12 comments

bbin4ry_d3struct0r

I'm wondering if this is a bug.

6 comments

bbin4ry_d3struct0r

Otherwise, what's the point?

11 comments

bbin4ry_d3struct0r

I've had to reduce `Settings.chunk_size`

I've had to reduce Settings.chunk_size by half as well as similarity_top_k in the query engine to get working what worked fairly well in v.0.9.

22 comments

bbin4ry_d3struct0r

Yay ... all my old code is now broken 🥳

11 comments

bbin4ry_d3struct0r

I would find it very hard to believe

I would find it very hard to believe that I'm the first one to think of such a basic idea, but I'm looking for examples of open source projects and haven't come across any.

8 comments

bbin4ry_d3struct0r

The nodes in my VectorStoreIndex have sequential identifiers. How do I add new nodes to th

@kapa.ai The nodes in my VectorStoreIndex have sequential identifiers. How do I add new nodes to the index with identifiers starting from the identifier of the last node in the index?

14 comments

bbin4ry_d3struct0r

Okay, nvm ... apparently, there _is_

Okay, nvm ... apparently, there is someone in the RAG space named Jason Liu who does actually own a Porsche, so it wasn't a shot against Jerry with his name misspelled.

1 comment

bbin4ry_d3struct0r

Bm25

Very strange!

6 comments

bbin4ry_d3struct0r

Noob question: What is the practical

Noob question: What is the practical difference between working with documents vs. working with nodes?

For example:

index = VectorStoreIndex.from_documents(documents, service_context = service_context)

vs.

index = VectorStoreIndex(nodes, service_context = service_context)

Are there significant (dis)advantages to each?

1 comment

Find answers from the community

Context window

Why similarity_top_k can be set directly when initializing a VectorStoreIndex as a query engine but not as a chat engine

My basic understanding of the

Without seeing a specific error output,

Query engine

So, if I want to build a bot that

I'm wondering if this is a bug.

Otherwise, what's the point?

I've had to reduce `Settings.chunk_size`

Yay ... all my old code is now broken 🥳

I would find it very hard to believe

The nodes in my VectorStoreIndex have sequential identifiers. How do I add new nodes to th

Okay, nvm ... apparently, there _is_

Bm25

Noob question: What is the practical