Hey what is the default prompt we re

Almost sure that’s this one

Also, it will use the refine prompt if the retrieved nodes don't all fit into the LLM input, to refine across multiple calls

https://github.com/jerryjliu/llama_index/blob/c86eb14b16779d5e845579a84a4704996ae467e5/llama_index/prompts/default_prompts.py#L81

Thank you! Also, do you happen to know if we can retrieve the final prompt that will be sent to GPT?

this is to handle InvalidRequestError: This model's maximum context length is... of openai when we exceed the token limits ?

Yea pretty much, llama-index does its best to avoid those errors

I have an article about it, maybe can help you https://dev.to/emanuelferreira/understanding-context-augmentation-behind-llama-index-for-beginners-jdl

You can turn on debug logs (I think this gets logged?) Or you can use the LlamaDebugHandler and inspect the LLM events

We also have a recent weights and biases integration
https://github.com/jerryjliu/llama_index/tree/main/docs/examples/callbacks

so at the moment we cannot do that in python like .get_final_promt() function?

Not really, but feel free to make a PR if you think that would be helpful 👍 might be a goof function for the llama debug handler

I'm curious what you need the final promot for? Just debugging?

yeah sure! the current system saves those kind of information in a db, so I'm trying to refactor that by using llama

And can we use a dynamic number of chunks returned? so instead of configuring similarity_top_k parameter of VectorIndexRetriever, it returns just the relevant ones ? (maybe using a threshold or s.th, not sure if there is already a feature for that)

You can set a similarity threshold yes (in my experience with OpenAI, you probably want the threshold at LEAST at 0.78). Then you can set the top_k to be larger and hope the threshold trims it down lol

https://gpt-index.readthedocs.io/en/latest/guides/primer/usage_pattern.html#low-level-api

I'll try that, thank you!

Hey, for the threshold, I just need to use this code, right?
response_synthesizer = ResponseSynthesizer.from_args( node_postprocessors=[ SimilarityPostprocessor(similarity_cutoff=0.7) ] )
because when I use it, I get an empty chunk list (I even tested with 0.1 and the same)