predict when synthezing, response = self._llm.complete(formatted_prompt) chat_response = self._llm.chat(messages) instead?`
def predict(
self,
prompt: BasePromptTemplate,
output_cls: Optional[BaseModel] = None,
**prompt_args: Any,
) -> str:
"""Predict."""
self._log_template_data(prompt, **prompt_args)
if output_cls is not None:
output = self._run_program(output_cls, prompt, **prompt_args)
elif self._llm.metadata.is_chat_model:
messages = prompt.format_messages(llm=self._llm, **prompt_args)
messages = self._extend_messages(messages)
chat_response = self._llm.chat(messages)
output = chat_response.message.content or ""
else:
formatted_prompt = prompt.format(llm=self._llm, **prompt_args)
formatted_prompt = self._extend_prompt(formatted_prompt)
response = self._llm.complete(formatted_prompt)
output = response.text
llm.predict() will use llm.complete() or llm.chat() depending on if the LLM is a chat model or not'/deployments/my-deployed-gpt-4-model/chat/completions'
chat that means it'll use the llm.chat() ?llm.complete / completions api should be kinda deprecated and OpenAI / Azure OpenAI has moved on to chat