Find answers from the community

Updated 2 months ago

OpenAI

Hi,

Quick question. I'm using OpenAIPydanticProgram, to get structured data from gpt4. How do I get from its output info about the call? I want to know, for exemple, outut and input tokens used. This is my code:

    program = OpenAIPydanticProgram.from_defaults(
        output_cls=ListItems,
        llm=llm,
        prompt=prompt,
        verbose=True,
    )
    output = program(capitulo=capitulo,posicao = posicao,exemplos = exemplos)

Can I get that kind of info from output?

3 comments

WWhiteFang_Jr

If you want to check on the tokens being used for every call , you can do so by doing this

Plain Text

import openai
openai.log = "debug"

mmwon

But that is logging only, no? I would like to have access from the output. Equivalent when we use directly chat.completions

WWhiteFang_Jr

Did not find anything related to token in the OpenAIPydanticProgram. But You can use the tokencounting handler from LlamaIndex to get the details for each call.
https://docs.llamaindex.ai/en/stable/examples/callbacks/TokenCountingHandler.html

Add a reply