Hi! π
I am running into an issue where if I set service_context=service_context in query_engine like so:
llm_predictor = ChatGPTLLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo", streaming=False))
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)
query_engine = index.as_query_engine(text_qa_template=CHAT_QA_PROMPT,
refine_template=CHAT_REFINE_PROMPT,
similarity_top_k=3,
streaming=False,
service_context=service_context)
then the chatGPT does NOT have access to General Knowledge.
However, when I do NOT set service_context=service_context in query_engine like so:
query_engine = index.as_query_engine(text_qa_template=CHAT_QA_PROMPT,
refine_template=CHAT_REFINE_PROMPT,
similarity_top_k=3)
then I do have access to General Knowledge, but the ChatGPT response is getting cut off when it writes longer text response.
How do I achieve both access to General Knowledge AND no cut off of longer text responses?
Thank you!