Hi,

JJG

Hi,
I'm trying to use the new Instrumentation module, but having difficulty adding a @dispatcher.span decorator to a generator function since the span ends as soon as a value is yielded.
I started to implement my custom span without the decorator, as shown in the module guide here https://docs.llamaindex.ai/en/stable/module_guides/observability/instrumentation/#enteringexiting-a-span
but looking at the source code, it seems some extra logic was added to the decorator for handling threads, and I'm not sure if I need to rewrite those locks in my code https://github.com/run-llama/llama_index/blob/baa3e82e56a647d0281135c8c279fa1c386e8f6c/llama-index-core/llama_index/core/instrumentation/dispatcher.py#L261-L265
It would be nice if LlamaIndex had a context manager for creating spans or if the decorator worked with generator functions.

8 comments

LLogan M

due to how decorators and generators work, its not easily possible to make the two compatible with just a generator

You'd need to call span_enter() and span_exit() on your own

LLogan M

If you see a way to make it work, I welcome a PR

JJG

If I call span_enter()/span_exit() do I also need to call:
with dispatcher.lock:
dispatcher.set_current_spanid(id)
?

LLogan M

eh, only if you are doing stuff with async/threading I think

LLogan M

this is to keep it async/thread safe

JJG

I'm using async/threads since it's part of an HTTP service and need to support multiple concurrent requests (currently using uvicorn).

JJG

I just did a hacky workaround for the decorator which seems to work. Basically just replaced this line

Plain Text

 result = await func(*args, **kwargs)

with

Plain Text

  async for result in func(*args, **kwargs):
     yield result

https://github.com/run-llama/llama_index/blob/baa3e82e56a647d0281135c8c279fa1c386e8f6c/llama-index-core/llama_index/core/instrumentation/dispatcher.py#L307

LLogan M

then you would need the lock statements 👀

Or the workaround works too

Add a reply

Find answers from the community

Hi,