Regarding Open AI costs when llama index

At a glance

The community member is using the LlamaIndex library to build a simple query endpoint and is concerned about the costs associated with the OpenAI API calls. They are unsure if the SimpleWebPageReader().load_data and the query request are made in a single call to OpenAI or multiple calls. Another community member suggests checking the OpenAI API page for specific cost information or calculating the costs based on the number of tokens used. The community members also discuss the possibility of caching the SimpleWebPageReader().load_data call and using the openai.log = "debug" feature to log the API requests.

Useful resources

PPunkbit

Regarding Open AI costs when llama index data loading and responding. For my use case I'm using a Restful API to access handle requests.

I've done a quick test of llama index as a simple query endpoint, let's say:

Plain Text

def query(question: Union[str, None] = None):
  documents = SimpleWebPageReader(html_to_text=True)
         .load_data(["https://docs.foobar.com/some-knowledge"])
  index = SummaryIndex.from_documents(documents)
  query_engine = index.as_query_engine() 
  answer = query_engine.query(question)

  return { "answer": str(answer )}

It can be determined that for every GET query request there's an associated cost.

Since each computation over Open AI has a cost, I would like to know how costly the operation is for the query endpoint above.

I'm assuming that the SimpleWebPageReader.load_data and query question goes on a single request to Open AI and not two or more?

7 comments

TTeemu

Depends on how many tokens it's using, you can also check your OpenAI API page for information about specific costs if you don't want to calculate it yourself based on the tokens used

@Logan M Do you know if the latest version supports openai.log = "debug" ?

PPunkbit

@Teemu thanks for looking! My main concern here would be to know if I'd rather cache the SimpleWebPageReader().load_data as I don't know what the process does. Would that make any difference you reckon?

TTeemu

Have you tried openai.log = "debug"

TTeemu

It should log what gets sent to the API

TTeemu

But I just updated to latest version and was having some issues with it

PPunkbit

@Teemu I'll check. Just found about typescript version, so switched to it and re-writing atm

LLogan M

(for reference, it does support that debug logging!)

Add a reply

Find answers from the community

Regarding Open AI costs when llama index