The post raises the issue of standardizing metadata in responses, as using raw response to count tokens may not scale across different providers. Community members suggest capturing metadata like input_tokens_count, output_tokens_count, stop_reason, and stop_sequence. They also discuss the possibility of exposing a TokenCounter in the handler, allowing each integration to have its own implementation, with defaults for missing metadata. The community members agree that this would be a useful long-term solution, especially for directed ReAct agents, and propose using glue code in the token counter to use the metadata if available, or fall back to the original logic.
Is there any plans for standardizing the metadata in responses? Picking raw response for example to count the tokens will not scale across the many different providers.
