I'm looking for an example of

ttimothybeamish

I'm looking for an example of Instrumentation with a custom Span. I'd like to create a custom span instance with a few properties and then enter that span at a specific point in our RAG pipeline, where I can assign values to those properties, and then exit it while updating some properties. I've read https://docs.llamaindex.ai/en/stable/module_guides/observability/instrumentation and looked at the linked notebooks, but it's unclear to me how we're meant to have our specific custom span used at a specific point in time. For example, when calling dispatcher.span_enter(...), I do not understand what values I'm meant to provide for id_ or bounded_args and I'm only guessing that instance is meant to be the instance of the span I want to use. Futhermore, I can't clearly see how my custom span is the one used when I use the @dispatcher.span decorator. I have placed breakpoints inside the new_span, prepare_to_exit_span, and prepare_to_drop_span functions inside my custom span but it doesn't look like they are called. An example would be helpful. Thanks.

12 comments

LLogan M

The @dispatcher.span decorator will dispatch a span for each span handler you attached to the dispatcher.

So you would create your custom spans and custom span handler, attach it, decorate your function, and off you go

Theres a lot of async/thread-safe logic that goes into the decorator. It calls span_enter and span_drop for you

ttimothybeamish

Span Handlers consume spans, right?

How do I attach a property to a specific Span such that, when I see it's "start" event in Phoenix, I can also see this property?

LLogan M

spans are mostly just containers for event boundaries

in like 99% of use-cases, I think you probably just want a custom event handler, or just a span handler that attaches some initial attributes (like session info or something) 🤔

To attach a property to a span, you'd have to manually call span_enter and span_drop

This is how the decorator calls it

Plain Text

@wrapt.decorator
def wrapper(func, instance, args, kwargs):
    bound_args = inspect.signature(func).bind(*args, **kwargs)
    id_ = f"{func.__qualname__}-{uuid.uuid4()}"

    token = active_span_id.set(id_)
    parent_id = None if token.old_value is Token.MISSING else token.old_value
    self.span_enter(
        id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    )
    try:
        result = func(*args, **kwargs)
    except BaseException as e:
        self.event(SpanDropEvent(span_id=id_, err_str=str(e)))
        self.span_drop(id_=id_, bound_args=bound_args, instance=instance, err=e)
        raise
    else:
        self.span_exit(
            id_=id_, bound_args=bound_args, instance=instance, result=result
        )
        return result
    finally:
        # clean up
        active_span_id.reset(token)

LLogan M

rip formatting, oh weel

ttimothybeamish

Phoenix only appears to log Spans. If I fire an event, I don't see it in Phoenix. Should I create an Event Handler that listens for 2 events (a start and an end) and then creates a custom span when it sees the second event?

I'm still not sure how to print custom properties to a span such that they show up in Phoenix. An example would be helpful

ttimothybeamish

I made this

Plain Text

class OrchestratorPipelineSpan(BaseSpan):
    num_of_agents: int = Field(default=0)
    agents_names: str = Field(default="nothing here")


class OrchestratorPipelineSpanHandler(BaseSpanHandler[OrchestratorPipelineSpan]):
    @classmethod
    def class_name(cls) -> str:
        """Class name."""
        return "OrchestratorPipelineSpanHandler"

    def new_span(
        self, id: str, parent_span_id: Optional[str], **kwargs
    ) -> Optional[OrchestratorPipelineSpan]:
        """Create a span."""
        # logic for creating a new MyCustomSpan
        print("new_span")
        return OrchestratorPipelineSpan(num_of_agents=42, agents_names="agent1, agent2")

    def prepare_to_exit_span(
        self, id: str, result: Optional[Any] = None, **kwargs
    ) -> Any:
        """Logic for preparing to exit a span."""
        print("prepare_to_exit_span")

    def prepare_to_drop_span(self, id: str, err: Optional[Exception], **kwargs) -> Any:
        """Logic for preparing to drop a span."""
        print("prepare_to_drop_span")


root_dispatcher = instrument.get_dispatcher()
root_dispatcher.add_span_handler(OrchestratorPipelineSpanHandler())

and I have breakpoints on each of those print statements in each function in the OrchestratorPipelineSpanHandler, but they never get hit. This is surprising.

LLogan M

I thiiiink events are only recorded if arize if they happen under a existing span 🤔 (Because arize itself is creating their own span handler)

LLogan M

let me make a quick example maybe

ttimothybeamish

I think you are correct - events are recorded if they happen under existing spans BUT these events also require a span_id property which is the ID of the open span you wish the event to be placed in.
(source: https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-llama-index/src/openinference/instrumentation/llama_index/_handler.py#L752)

How can we easily get a span_id for a span that is auto-generated via the @dispatcher.span decorator?

ttimothybeamish

I see a current_span_ids property inside the Dispatcher but I don't see it used (yet). It's also a dictionary so it's unclear which span id inside this dictionary is the current active span.

LLogan M

The decorator merely generates the id to be id_ = f"{func.__qualname__}-{uuid.uuid4()}" and then keeps track of the parent span 🤔 So youd have to repliacte the decorators method for keeping track of a parent span id? I thin?

ttimothybeamish

I'm a little lost on how to log an event.

We do

Plain Text

dispatcher = instrument.get_dispatcher(__name__)
dispatcher.event(some_custom_event)

And I can trace that to process _event where it calls self ._process _event that appears to do nothing but log a warning and then returns on line 295 because we are not waiting for streaming.

Am I meant to create my own event handler that turns an event into a span?

Add a reply

Find answers from the community

I'm looking for an example of