hi @motoki , first thing is why is max_input_size set so high? davinci only goes up to 4000 tokens
also, num_output needs to be less than max_input_size, since max_input_size includes num_output
I had mistakenly thought that max_input_size and num_output were the overall length of the input documents and the overall length of the text to be output.
Thank you for pointing this out.
However, even if I reduce the size as shown below, the query results are interrupted in the middle.
pf = PromptHelper(
max_input_size=4000,
num_output=2000,
chunk_size_limit=2000,
max_chunk_overlap=0,
separator=".
)
I also found that if the query is written to output in Japanese, it is missing in the middle, and if it is output in English, everything is output correctly.
ooooh interesting. when you say the query results are interrupted you mean the output is truncated?
there's not an easy way to prevent that with openai's api
interesting that it happens for japanese though and not english
I modified num_output as follows and now I get all results in Japanese.
Thank you very much!
pf = PromptHelper(
max_input_size=4000,
num_output=3000,
chunk_size_limit=1000,
max_chunk_overlap=0,
separator="\n\n"
)
set number of output tokens
num_output = 1024
llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-davinci-003", max_tokens=num_output))
index = GPTSimpleVectorIndex(
documents = documents,
prompt_helper = pf,
llm_predictor=llm_predictor
)I have a question about using an additionally created index as one of the TOOLS in a langchain.
If I do the following, the output will be different when I run index.query('query_str') and when I run agent_chain.run(input='query_str').
How can I modify this to make them match?
item_ap_index = GPTSimpleVectorIndex.load_from_disk('index1.json')
ツールの準備
tools = [
Tool(
name = "index1",
func=lambda q: str(index1.query(q)),
description="""answer the question in japanese
""",
return_direct=True,
),
]
set Logging to DEBUG for more detailed outputs
memory = ConversationBufferMemory(memory_key="chat_history")
llm=OpenAI(temperature=0)
from langchain.callbacks.base import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
agent_chain = initialize_agent(
tools=tools,
llm=OpenAI(max_tokens=1024, temperature=0, streaming=True, callback_manager=CallbackManager([StreamingStdOutCallbackHandler()])),
agnet='zero-shot-react-description',
verbose=True,
memory=ConversationBufferMemory(memory_key="chat_history")
)
agent_chain.run(
input="""'query_str"""
)@motoki is the agent using the gpt index tool? or is it choosing to not use it?
@jerryjliu0 in my case the langchain agent is choosing to not use the tool, is there any way to force the agent to use it? Thanks!
yeah @ballinho that may just require a bit of prompt engineering on the description you set for the Tool, as well as the agent template, to convince the agent to use it. i also had to fiddle a bit with the tool description to convince the agent to use it
