How can I get the debug info shown when

How can I get the debug info shown when setting verbose=True through the code?

11 comments

Try also setting this at the top of your code

Plain Text

import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

yyoelk

Thanks, I did manage to see the prints. My question is how to access them through the code if I want to present some of them in my web app

yyoelk

For example, if a query got transformed to another query, I want to show the user the transformed version too. Or, if I wanted to compare multiple documents which got decomposed to several separate queries for each document

LLogan M

hmmm, not an easy way to do that at the moment.

You can try adding an additional logger handler, that is a custom class designed to grab the logs and do something with them. The python logging object supports multiple handlers, and each handler is called for each log

LLogan M

Or you can do something else to capture stdout, there might be some other method 🙂

LLogan M

https://stackoverflow.com/questions/3118059/how-to-write-custom-python-logging-handler

yyoelk

For now I thought just redirecting those verbose outputs to a file and then try to parse it, but obviously would have been much better if all those middle steps were saved in python objects

LLogan M

The logging handler would definitely make that possible and easier/more customizable yea. There might be something else too, just need to check the code

LLogan M

So, using the LlamaDebugHandler, technically the query transforms will be captured as LLM events.

The only tricky part is you'd have to iterate over the LLM events and remove any that aren't using the query transform template

https://github.com/jerryjliu/llama_index/blob/main/docs/examples/callbacks/LlamaDebugHandler.ipynb

You'd want to do event_pairs = llama_debug.get_event_pairs(CBEventType.LLM) to get the LLM events after the query, then optionally use flush_event_logs() to clear the events

LLogan M

The LLM events will log the templates + template args on event start, and response + formatted prompt on event end.

yyoelk

Thanks, will look into it

Add a reply

Find answers from the community

How can I get the debug info shown when