Find answers from the community

Updated 3 months ago

Token error

I am getting the following error (inidicating that i am exceding token limits):
Plain Text
INFO:openai:error_code=None error_message="This model's maximum context length is 4097 tokens, however you requested 4358 tokens (3334 in your prompt; 1024 for the completion). Please reduce your prompt; or completion length." error_param=None error_type=invalid_request_error message='OpenAI API error received' stream_error=False


I thought i had the llm to use a model with more available tokens (gpt-3.5-turbo-16k). This i how i started it :

Plain Text
num_outputs = 1024
llm_predictor = LLMPredictor(
    llm=OpenAI(
        temperature=0.1,
        model_name="gpt-3.5-turbo-16k",
        max_tokens=num_outputs,
        streaming=True,
    )
)


storage_context = StorageContext.from_defaults(persist_dir="indexstore/newnew")
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)
# print(storage_context)
index = load_index_from_storage(
    storage_context, service_context=service_context
)  # Load the index


I am initializing it wrong? I get this error sometimes only, not all the time.
L
F
2 comments
Try using the LLM from llama_index, not langchain

Plain Text
from llama_index.llms import OpenAI

service_context = ServiceContext.from_defaults(llm=OpenAI(model="gpt-3.5-turbo-16k"  temperature=0, max_tokens=1000))
Yep solved the issue!!
Add a reply
Sign up and join the conversation on Discord