Refactoring code for chat engine functionality

SSaltuk

I wanted to use my knowledge base that's why i'm calling my own chat_engine function. I just refactored the code to use a query engine instead of an Agent-based approach for the finetuning_events. That should work, but still weird that it fails because of such a reason. I think there needs to be some fixing to be done to the ChatCompletionMessageToolCall class?

22 comments

LLogan M

need to serialize all the chat messages imo (the handler probably hasn't been updated since openai started returning pyndatic objects in their client)

SSaltuk

So nothing to be done from my side?

SSaltuk

Except of creating a PR maybe 🙂

LLogan M

You can probably iterate over handler._finetuning_events and handler._function_events and make sure they are all dicts

LLogan M

actually that won't work

LLogan M

likely needs a PR yea

SSaltuk

I have one more question. I just let the function get_chat_engine return a query engine. And now i don't get the error anymore but save_finetuning_events is empty so has a length of zero?

SSaltuk

DO you know what is wrong there?

SSaltuk

Attachment

LLogan M

maybe the callback manager wasn't properly attached to the llm in that case?

SSaltuk

Why do i need to attach it to the llm?

SSaltuk

Seems weird

SSaltuk

There is also the Settings.callback_manager option directly

SSaltuk

unofrtunately doesn't work also when i try Settings.llm.callback_manager =

LLogan M

Because the callback manager is tied to the llm? Thats how it collects the events?

Plain Text

llm = OpenAI(..., callback_manager=callback_manager)
query_engine = index.as_query_engine(llm=llm)

SSaltuk

I would have assumed that using the Settings.callback_manager automatically resolves the llm appropriately?

SSaltuk

Even when i try it in this way, for some reason it is not being attached properly

LLogan M

Not sure man 🤷 I would have to reproduce

SSaltuk

I think the issue happens when i use the ResponseMode.CONTEXT_ONLY. What is the problem with it?

SSaltuk

So when i use this mode and want to log finetuning events i get this message:

Plain Text

Wrote 0 examples to finetuning_events.jsonl

LLogan M

Oh context only doesn't call the llm

LLogan M

So there is no event to record

Add a reply

Find answers from the community

Refactoring code for chat engine functionality