Find answers from the community

Updated 2 months ago

Hi,

Hi,
I'm trying to use the new Instrumentation module, but having difficulty adding a @dispatcher.span decorator to a generator function since the span ends as soon as a value is yielded.
I started to implement my custom span without the decorator, as shown in the module guide here https://docs.llamaindex.ai/en/stable/module_guides/observability/instrumentation/#enteringexiting-a-span
but looking at the source code, it seems some extra logic was added to the decorator for handling threads, and I'm not sure if I need to rewrite those locks in my code https://github.com/run-llama/llama_index/blob/baa3e82e56a647d0281135c8c279fa1c386e8f6c/llama-index-core/llama_index/core/instrumentation/dispatcher.py#L261-L265
It would be nice if LlamaIndex had a context manager for creating spans or if the decorator worked with generator functions.
L
J
8 comments
due to how decorators and generators work, its not easily possible to make the two compatible with just a generator

You'd need to call span_enter() and span_exit() on your own
If you see a way to make it work, I welcome a PR
If I call span_enter()/span_exit() do I also need to call:
with dispatcher.lock:
dispatcher.set_current_spanid(id)
?
eh, only if you are doing stuff with async/threading I think
this is to keep it async/thread safe
I'm using async/threads since it's part of an HTTP service and need to support multiple concurrent requests (currently using uvicorn).
I just did a hacky workaround for the decorator which seems to work. Basically just replaced this line
Plain Text
 result = await func(*args, **kwargs)

with
Plain Text
  async for result in func(*args, **kwargs):
     yield result

https://github.com/run-llama/llama_index/blob/baa3e82e56a647d0281135c8c279fa1c386e8f6c/llama-index-core/llama_index/core/instrumentation/dispatcher.py#L307
then you would need the lock statements πŸ‘€

Or the workaround works too
Add a reply
Sign up and join the conversation on Discord