Source nodes

I think my previous response had a good example 👍

Was there something more specific you wanted to see?

To check the similarity scores, you can do something like response.source_nodes[0].score

(I think lol)

But it was not as stated.

So I asked a question from bot that was not in my text (from file). Example: "How to create telegram api". After that (bot) answered me

Please see this images. There is nothing about "create telegram api" in my document (.txt file). How does it answer that?

@Logan M @V

🤔

Thank you in advance

🙂

Right. So internally llama index tells the model to only use information from your documents (and not from itself) to answer queries, but the model doesn't always follow instructions. (They aren't as smart as us yet haha).

So there are three options in this case.

1. Check response.source_nodes to ensure that the text use to generate the response has sufficient similarity

1. Set something like similarity_cutoff=0.6 to filter out documents that don't relate to the query

1. Do some prompt engineering and customize the internal prompts:

https://gpt-index.readthedocs.io/en/latest/how_to/customization/custom_prompts.html

As far as I know, This program is looking for answers in GPT that it can't find in the doc. Can't we take a different action against this now? Maybe I don't want to make requests to GPT 🙂

It's a little tricky! I think the similarity_cutoff (to remoce unrelated nodes) or prompt engineering (to better enforce how the LLM acts and responds) is the best option.

Or, you can set the response mode to only fetch the source nodes and not call the LLM

index.query(..., response_mode="no_text")

As far as I know there are 3 types:

default
compact
tree_summarize

https://gpt-index.readthedocs.io/en/latest/guides/primer/usage_pattern.html#setting-response-mode

But you say me: "no_text"

🤔

It's a bit of a hidden mode. Try it out

😅

(Source code definition)

https://github.com/jerryjliu/llama_index/blob/main/gpt_index/indices/response/builder.py#L30

I saw something about nodes in your doc. Is there any good article in medium or another place you can recommend about this?

Which nodes? The source nodes?

The best place for info is searching in the docs I think (or reading source code lol), I haven't really tried looking up any articles 😅

I was saying these.

🙂

Oh I see!

When you create an index with documents, they get broken down into nodes.

You can actually break the documents into nodes ahead of time too! Either manually or with a function

https://gpt-index.readthedocs.io/en/latest/guides/primer/usage_pattern.html#parse-the-documents-into-nodes

In the future, I think these nodes will be more important. Keep your eyes out for updates 💪

Thank you for the information