is it included in the embedding if I use

At a glance

The post asks whether the LLM (Large Language Model) includes the content in the embedding, and whether it just appends the title to the node text before embedding the chunk. The comments indicate that the content is included in the embedding, and provide some details on how to customize the metadata inclusion in the embeddings. One community member suggests using metadataMode='embed' and excludedEmbedMetadataKeys to control which metadata is included in the embeddings. However, there is no explicitly marked answer to the original question.

Useful resources

sshakedbuk

is it included in the embedding if I use the llm to embed it? , does it just append it to the node text before embedding the chunk+ title

5 comments

LLogan M

yes and yes

You can see more details here
https://docs.llamaindex.ai/en/stable/module_guides/loading/documents_and_nodes/usage_documents.html#advanced-metadata-customization

sshakedbuk

      let titleExtractor= new TitleExtractor({llm:azureOpenAillm})
      let nodes = await titleExtractor.transform(titleExtractedNodes)
      
      let nodesWithMetadata = []

      for(let node of nodes){
          nodesWithMetadata.push(node.getContent())
      }

sshakedbuk

when I use it like this I also get the "documentTitle:" part inside the vector, how to remove it?

sshakedbuk

node.getContent({metadataMode:'all'}) using this does not work

LLogan M

Losing me a bit with that code there.

In any case, each piece of metadata can be included or not included in embeddings. Use metadataMode "embed", and set node.excludedEmbedMetadataKeys =[..] to a list of keys to exclude

Add a reply

Find answers from the community

is it included in the embedding if I use