Find answers from the community

Updated 2 months ago

Hi I found a strange problem If a

Hi, I found a strange problem. If a question is asked in a launguage different from English, the response somtimes is cut in the middle of word if it's not very short. For example, I ask a question and response is cut, its length is 251 symbols. Not sure why is that and how to fix it. Thanks!
L
S
2 comments
Yea, languages other than english tend to use more tokens, and the default max_tokens from OpenAI is 256

You can increase the max_tokens by setting both max_tokens and num_output as seen here

Plain Text
from llama_index import ServiceContext, LLMPredictor, VectorStoreIndex
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI

# use gpt-3?
llm = OpenAI(model_name="text-davinci-003", temperature=0, max_tokens=512)

# use gpt-3.5 or gpt-4?
llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0, max_tokens=512)

llm_predictor = LLMPredictor(llm=llm)

service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, num_output=512)

index = VectorStoreIndex.from_document(documents, service_context=service_context)
Thanks a lot, will do it!
Add a reply
Sign up and join the conversation on Discord