Find answers from the community

Updated 3 months ago

Logan cosplay

I’m a little confused on the nodes v embeddings. I notice better search quires when using embeddings but I’m wondering how llama index enhances the the retrieval when vectors data bases are implemented

11 comments

iisaackogan

Hey

iisaackogan

So basically, Nodes are text chunks of data @DangFutures

iisaackogan

Arbitrarily sized based on your preference. The standard is each sentence is a chunk/node of data

iisaackogan

The way that embedding works is it converts each chunk into a vector representation

iisaackogan

Then, when you send a prompt, your prompt is converted to a vector representation

iisaackogan

Then, the distance between the two vectors is calculated (K-distance)

iisaackogan

What this means is that using LlamaIndex, you can reduce the amount of information from your data and also make sure the information is RELEVANT to the prompt

iisaackogan

Because you are calculating the mathematical difference between your prompt and each node in the vector database, which gets you only the most relevant data (top_k)

iisaackogan

Does that help?

DDangFutures

Holy moly thank you!!!

iisaackogan

No problem @DangFutures

Add a reply