Hi! How can I set up the OpenAI

SSeaCat

Hi! How can I set up the OpenAI temperature? Despite I've set it up explicitly, in the debug it's always 0.1. Here is the code:

Plain Text

llm_engine = OpenAI(temperature=0, model=model_name, api_key=openai_key, max_tokens=response_limit, callback_manager=callback_manager)

Are there other places to set it up? Thanks!

2 comments

LLogan M

Its working fine

Plain Text

>>> from llama_index.llms.openai import OpenAI
>>> llm = OpenAI(temperature=0)
>>> llm.temperature
0.0
>>> llm._get_model_kwargs()
{'model': 'gpt-3.5-turbo', 'temperature': 0.0}
>>>

But perhaps some component in your system is defaulting to the default LLM. What do you with that llm variable?

SSeaCat

I use it here:

Plain Text

query_engine = index.as_chat_engine(chat_mode='context', 
                     similarity_top_k=similarity_top_k, 
                     llm=llm_engine,
                     system_prompt=prepared_system_prompt)

And because I have the env var showing the OpenAI debugging information, I can see in the terminal this:

Plain Text

Request options: {'method': 'post', 'url': '/chat/completions', 'files': None, 'json_data': {'messages': [{'role': 'system', 'content': '---------------------\nBelow is private information: '}, {'role': 'user', 'content': 'pricing'}], 'model': 'gpt-3.5-turbo', 'stream': False, 'temperature': 0.1}}

Add a reply

Find answers from the community

Hi! How can I set up the OpenAI