Find answers from the community

Updated 9 months ago

I suppose this is a bug, just wanted to

I suppose this is a bug, just wanted to double check before reporting it
L
Ł
10 comments
yea I think this is a bug lol not sure if its promptlayer or llama-index, I havent dug into it
I spoke to the promptlayer author, he's not totally sure either 😛 From my debugging I'm pretty sure this is to be fixed within the llama_index integration
As it's probably just a matter of including more events for tracing
Right now only strict 'LLM' calls are tracked
does prompt layer support more than LLM calls? (its been a hot minute since ive used it haha)
If you are up to, I will definitely review/merge any PRs 🙂
I could give it a shot, however that's the thing - llama_index is making calls of type CHUNKED which I have too little knowledge to understand where it's coming from - afaik I'm just making simple LLM calls with that pseudocode I just showed
The response synthesizer by default is "compact" -- this means it combines all retrieved nodes into one chunk, and then splits again, so that each LLM input as big as possible (this reduces overall LLM calls)
Ah, I see makes sense
In that case I'll give it a shot when I have a moment
Add a reply
Sign up and join the conversation on Discord