@Logan M Have you seen this error before

At a glance

A community member is experiencing an error with their llama-index update, where the "Calculated available context size -4316 was not non-negative". Other community members suggest this is related to the LLM/service context settings, specifically the max_tokens parameter being set to 8192 for the GPT-3.5-turbo model, which has a context window of only 4096. This appears to be the issue causing the error.

BBioHacker

Have you seen this error before?
Calculated available context size -4316 was not non-negative
I just updated my llama-index and am getting this.

11 comments

LLogan M

Usually that's related to some odd llm/service context settings

BBioHacker

we are using openai and everything was working yesterday

BBioHacker

but it got messed up right now as we update to 0.9.39

LLogan M

can you share some llm/service context code?

LLogan M

I can probably spot the issue pretty quickly

BBioHacker

yep just a second

dddashed

def get_llm(openai_api_key, max_tokens=8192):

os.environ["OPENAI_API_KEY"] = openai_api_key
return OpenAI(
temperature=0.0, model='gpt-3.5-turbo', max_tokens=max_tokens
) def get_ds_index1(documents, llm, c_m, api_key):
# Defining the service context
embed_model = OpenAIEmbedding()
service_context = ServiceContext.from_defaults(
llm=llm,
chunk_size=384,
embed_model=embed_model,
callback_manager=c_m,
chunk_overlap=128
)

response_synthesizer = get_response_synthesizer(
response_mode="tree_summarize", use_async=False
)

# Processing the entire 'documents'
temp_index = DocumentSummaryIndex.from_documents(
documents,
service_context=service_context,
response_synthesizer=response_synthesizer
)

return temp_index

LLogan M

so you've set max_tokens=8192 for gpt-3.5-turbo, but that model only has a 4096 context window 🤔

Attachment

LLogan M

That would be the issue

BBioHacker

ahhh man sorry @Logan M

LLogan M

no worries 😎

Add a reply

Find answers from the community

@Logan M Have you seen this error before