Hey guys! I have a custom retriever that

SSriram Vadlamani

Hey guys! I have a custom retriever that uses another LLM to generate Nested Elasticsearch queries and returns a List[NodeWithScore] as a return type where NodeWithScore implements the base class BaseNode.
On this returned list of nodes, I want to perform a similarity top k, so now should I build a new index from this list of nodes? I don't really know how what to do from here, so I thought I'd ask for some advice.

31 comments

LLogan M

Yes you could make an index on the fly with these nodes

Plain Text

nodes = [x.node for x in elastic_nodes]
index = VectorStoreIndex(nodes=nodes)
retriever = index.as_retriever(similarity_top_k=2)
source_nodes = retriever.retrieve("query")

SSriram Vadlamani

oh. okay. Well thanks!

SSriram Vadlamani

and I'm guessing, since we have a retriever, we can also specify a query engine starting from this and also get the actual response itself right?

LLogan M

Oh, if you want the query engine, you could also do index.as_query_engine(similarity_top_k=2)

SSriram Vadlamani

ah that works? ok that's awsome

SSriram Vadlamani

it's timing out 😦

SSriram Vadlamani

Attachment

SSriram Vadlamani

wait when we make this new index on the fly, does it compute the embeddings again? cuz I already have the embeddings in the nodes

LLogan M

If the embeddings are attached to the nodes, it will not re-compute

LLogan M

Seems like its failing on the LLM call (not embeddings)

What LLM are you using?

LLogan M

ah its ollama

LLogan M

try increasing the timeout: llm = Ollama(..., request_timeout=3000)

SSriram Vadlamani

I tried with 120 and it was still the same. But lmao 3000 haha

SSriram Vadlamani

Cuz when I include the storage context variable in the index , the response takes about 15 seconds to generate

SSriram Vadlamani

But because I mentioned it. It reindexes the documents when I checked my Open search with postman

SSriram Vadlamani

Which I found very weird.

SSriram Vadlamani

I will try with 3k and let you know what that does.

LLogan M

It still re-embeds? Are you using VectorStoreIndex.from_documents() or VectorStoreIndex(nodes=nodes) ? The latter should not re-embed if each node has its node.embedding field populated

LLogan M

Ollama can be slow sometimes lol

SSriram Vadlamani

I'm using the constructor and not from_documents

LLogan M

hmmm. And node.embedding is populated for each node?

SSriram Vadlamani

Well I got the results of Open search from a requests.post. so the result is in json
So for each result, I did this:

Plain Text

nodes = []
for result in results:
    node = TextNode(text=result["text"], embedding=result["embedding"])
    nodes.append(node)

LLogan M

Weird. There is a loop in the code that shouldn't generate embeddings then if they are actually there. Very sus

Attachment

SSriram Vadlamani

Yeah I saw that. Which is why I found it weird as well.

SSriram Vadlamani

I'm gonna try looking into it in detail. With like a debugger tool or something

SSriram Vadlamani

I'm doing this for a project in a company and the deadline is Friday. I'm so ded lmao.

LLogan M

nah you got this lol

LLogan M

It will be emebdding the query text though (maybe thats the embedding call you saw?)

SSriram Vadlamani

Yeah there's a call for that as well but I was able to trace that. The other ones and the Ollama timeout when not mentioning a vector store .. that I can't understand

LLogan M

What code block exactly is causing confusion? Just to clarify?

SSriram Vadlamani

so I have two versions of my code starting from this list of nodes.
Version 1:

Plain Text

index = VectorStoreIndex(nodes, storage_context=OpenSearchVectorStore(client=client), service_context=ServiceContext.from_defaults(llm=Ollama(model="mistral"))) # client is of type OpenSearchVectorClient with my config inside
query_engine = index.as_query_engine(similarity_top_k=5)
query_engine.query("Random question")

Version 2:

Plain Text

index = VectorStoreIndex(nodes, service_context=ServiceContext.from_defaults(llm=Ollama(model="mistral")))
query_engine = index.as_query_engine(similarity_top_k=5)
query_engine.query("Random question")

Version 1: Works fine, gives a response in 15 seconds. (including node retrieval from opensearch that I mentioned) but adds 5 new documents to my OpenSearch database.

Version 2: times out

Add a reply

Find answers from the community

Hey guys! I have a custom retriever that