Find answers from the community

K
Kia
Offline, last seen 3 months ago
Joined September 25, 2024
Does Llama index track tokens / usage? and can we get any metrics for usage
1 comment
M
Would someone be able to assist here? I am trying to use the OpenAILike class with vLLM and am not sure where it makes this call to vLLM and have tried searching for a while. Would someone be able to assist? https://github.com/run-llama/llama_index/blob/main/llama_index/llms/openai_like.py
12 comments
L
K
@jerryjliu0 Quick question: I am currently using this model: llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.9, model_name="text-davinci-003"))

define prompt helper

set maximum input size

max_input_size = 4000096

set number of output tokens

num_output = 4000096

set maximum chunk overlap

max_chunk_overlap = -2000
prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)
But my output is being cut off mid-way
Am I missing something? The output is a five answer quiz so not terribly long
8 comments
j
M
H