Hi everyone I m just getting started

is this using chat gpt?

or text-davinci-003

if the former, this is a known issue with chatgpt api. sometimes it struggles with following more complex instructions

I'm using the chat gpt API. I'm able to see the intermediate prompts and responses by setting the logging level to debug, however it's quite messy, as it includes a lot of extra information. I'd like to add a verbose option so that the output would be something like:

Initial response: ...
Refine context: (snippet, not full text)
Refined response: Return the original answer. The provided context does not relate to ...

@Martok good point. will take a look at adding!

I achieved what I was trying to do initially by using a custom StringIO logger to grab the debug output, and then filtering each line. In order to make it work I had to modify /gpt_index/response/utils.py and use .strip() on response_text to remove excess newlines which are sometimes being added in front of the response somewhere along the line.

I'm still getting some newlines thrown in there sometimes, like this:

DEBUG:root:> Refined response:

Refined answer: ....

Instead of modifying utils.py, adding this after line 125 in /gpt_index/indices/response/builder.py seems to work better.

response = response.strip()

However, I'm unsure of how to handle newlines if they appear in the middle of the generated responses, which does occasionally happen.

I see....to clarify, is this purely for logging purposes? or are you trying to strip newlines from the response text for another purpose?

Well, using the debug output was just a relatively easy way to get the intermediate responses, and removing newlines is only needed because my code is reading the log output line by line looking for "Initial response:" and "Refined response:" Ideally, I think any intermediate responses should be stored in the Response object, since the final answer is not always helpful after going through the refining process.

got it, thanks

this is something we'll look into adding very soon!

Awesome!

@Martok sorry for the delay. this is a very initial attempt towards solving this: https://github.com/jerryjliu/llama_index/pull/745. just do from gpt_index.logger import LlamaLogger and add it during the index.query call. Currently only works during create/refine response synthesis (which is what you wanted to see)

Thanks!