Token error

I am getting the following error (inidicating that i am exceding token limits):

Plain Text

INFO:openai:error_code=None error_message="This model's maximum context length is 4097 tokens, however you requested 4358 tokens (3334 in your prompt; 1024 for the completion). Please reduce your prompt; or completion length." error_param=None error_type=invalid_request_error message='OpenAI API error received' stream_error=False

I thought i had the llm to use a model with more available tokens (gpt-3.5-turbo-16k). This i how i started it :

Plain Text

num_outputs = 1024
llm_predictor = LLMPredictor(
    llm=OpenAI(
        temperature=0.1,
        model_name="gpt-3.5-turbo-16k",
        max_tokens=num_outputs,
        streaming=True,
    )
)


storage_context = StorageContext.from_defaults(persist_dir="indexstore/newnew")
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)
# print(storage_context)
index = load_index_from_storage(
    storage_context, service_context=service_context
)  # Load the index

I am initializing it wrong? I get this error sometimes only, not all the time.

Find answers from the community

Token error