With instrumentation deprecated

At a glance

The community members are discussing the implications of the deprecated callbacks and the introduction of the BaseEventHandler in the llama_index.core.instrumentation.event_handlers. They note that the callback manager is still available, but a new instrumentation system is being implemented to replace it. The idea is to create an event handler or span handler to hook into events and spans happening under the hood, or even add custom events and spans. There is an example provided in the documentation that demonstrates how to observe API calls using the instrumentation system. The community members indicate that this is still a work in progress, but the eventual goal is to have the instrumentation system replace the callback system.

Useful resources

SSimon 📐🛠

With instrumentation deprecated callbacks what does this mean for the

Plain Text

CallbackManager

? For instance I use a callback manager to surface function calls made by agents as a query is being processed. How would I do this with the

Plain Text

BaseEventHandler

in the

Plain Text

llama_index.core.instrumentation.event_handlers

? Is a callback manager used at all when creating an index with callbacks being deprecated?

4 comments

LLogan M

callbacks/callback manager is still there. Instrumentation is a sort of parallel system being implemented to replace callbacks

its still sort of in-progress, but the idea is you create an event handler or span handler to hook into events/spans happening under the hood (or even adding your own events/spans)

SSimon 📐🛠

It would be really helpful to see how these both can coexist in an example.
I guess is the BaseEventHandler uncovering everything, callbacks included, and then its just a matter of queuing this BaseEvent for all events even agent function calls?

LLogan M

In a sense yea. We are still expanding the types of payloads for events being logged, but yea

There is an example here
https://docs.llamaindex.ai/en/stable/examples/instrumentation/observe_api_calls/

LLogan M

Still a bit of a work in prgress, but the eventual idea is to have this replace the callback system

Add a reply

Find answers from the community

With instrumentation deprecated