Find answers from the community

Updated 2 months ago

[Question]: How to print the final promp...

Hi, I'm trying to get the final processed prompt being sent to LLM. I searched online and noticed a related post answered by @Logan M : https://github.com/run-llama/llama_index/issues/13310
The code provided is:
from typing import Dict, List


from llama_index.core.instrumentation.events.llm import (
LLMChatEndEvent,
LLMChatStartEvent,
LLMChatInProgressEvent,
)


class ExampleEventHandler(BaseEventHandler):
events: List[BaseEvent] = []

@classmethod
def classname(cls) -> str: """Class name.""" return "ExampleEventHandler" def handle(self) -> None: """Logic for handling event.""" print("-----------------------") # all events have these attributes print(event.id)
print(event.timestamp)
print(event.span_id)

# event specific attributes
if isinstance(event, LLMChatStartEvent):
# initial
print(event.messages)
print(event.additional_kwargs)
print(event.model_dict)
elif isinstance(event, LLMChatInProgressEvent):
# streaming
print(event.response.delta)
elif isinstance(event, LLMChatEndEvent):
# final response
print(event.response)

self.events.append(event)
print("-----------------------")


import llama_index.core.instrumentation as instrument

dispatcher = instrument.get_dispatcher(name)
dispatcher.add_event_handler(ExampleEventHandler())


However, I'm confused about how to incorporate this code into mine. Basically I have a vector store index built from nodes, and I'm using it as query engine to ask questions.

from llama_index.core import VectorStoreIndex

recursive_index = VectorStoreIndex(nodes=base_nodes + objects)
recursive_query_engine = recursive_index.as_query_engine(
similarity_top_k=5,
verbose=True,
response_mode="compact"
)

Thanks for your help!
L
l
11 comments
That code already plugs directly into yours?
Just run that dispatcher code at the top before creating your query engine/index
Don't know if I miss anything, but when I ask questions to the query engine, it only shows the response without any logging. Here's my code.
do I need to initiate logging or something to see the prompt sent to LLM? Sorry if I asked dummy questions, and I really appreciate your help.
Hmm, I made this change

Plain Text
dispatcher = instrument.get_dispatcher()
dispatcher.add_event_handler(ExampleEventHandler())


And that was the change that made it work
seems like you remove the name inside the parentheses. I tried to do the same, but there's no logging of LLM prompt during query-asking.
Note that the code has a print_source_nodes(response) function, which prints out the texts of the top 5 retrieved nodes. However, the printed results are not the final prompt sent to LLM.
you may comment out the last line: print_source_nodes(res1). Now you may only see the logging from asking the query.
Works fine for me on Google colab
ah, looks like your handle() function under the self defined ExampleEventHandler class is different from mine. My code forgets to include event: BaseEvent as an input. I also added a bunch of "flush=True" as shown in your code. Now looks like it's working! Thank you so much for your help!
Add a reply
Sign up and join the conversation on Discord