Error:
File "/opt/conda/lib/python3.10/site-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/opt/conda/lib/python3.10/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/opt/conda/lib/python3.10/site-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
File "/opt/conda/lib/python3.10/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/conda/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/workspace/LLama-Hub/GpuRequests.py", line 177, in post
response = agent_chain.run(input=question)
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 216, in run
return self(kwargs)[self.output_keys[0]]
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 106, in __call__
inputs = self.prep_inputs(inputs)
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 195, in prep_inputs
self._validate_inputs(inputs)
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 75, in _validate_inputs
raise ValueError(f"Missing some input keys: {missing_keys}")
ValueError: Missing some input keys: {'instruction'}Try agent.run(instruction=question)
fixed that now there is another problem @Logan MCode 1:
index = GPTSimpleVectorIndex.from_documents(
documents,service_context=service_context
)
prefix = """Have a conversation with a human,answering the following questions as best you can. You have access to the following tools:"""
suffix = """Begin!"
Question: {input}
{agent_scratchpad}"""
prompt = ZeroShotAgent.create_prompt(
tools,
prefix=prefix,
suffix=suffix,
input_variables=["input", "agent_scratchpad"]
)
llm_chain = LLMChain(llm=llmPipeline, prompt=prompt)
agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)
agent_chain = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True, memory=memory)
response = agent_chain.run(input=question) Error:
response = agent_chain.run(input=question)
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 216, in run
return self(kwargs)[self.output_keys[0]]
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 116, in __call__
raise e
File "/opt/conda/lib/python3.10/site-packages/langchain/chains/base.py", line 113, in __call__
outputs = self._call(inputs)
File "/opt/conda/lib/python3.10/site-packages/langchain/agents/agent.py", line 792, in _call
next_step_output = self._take_next_step(
File "/opt/conda/lib/python3.10/site-packages/langchain/agents/agent.py", line 672, in _take_next_step
output = self.agent.plan(intermediate_steps, **inputs)
File "/opt/conda/lib/python3.10/site-packages/langchain/agents/agent.py", line 385, in plan
return self.output_parser.parse(full_output)
File "/opt/conda/lib/python3.10/site-packages/langchain/agents/mrkl/output_parser.py", line 20, in parse
raise ValueError(f"Could not parse LLM output: `{text}`")
ValueError: Could not parse LLM output: ``
More context for you before the other code 1 i do this 😄
llmPipeline = HuggingFacePipeline(pipeline=model_pipeline)
llm_predictor = LLMPredictor(llm=CustomLLM())
embed_model = LangchainEmbedding(HuggingFaceEmbeddings())
node_parser = SimpleNodeParser(text_splitter=TokenTextSplitter(chunk_size=512, chunk_overlap=max_chunk_overlap))
prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, embed_model=embed_model,
prompt_helper=prompt_helper, node_parser=node_parser,
chunk_size_limit=512)So i make a pipeline and separate llm_predictor but later I only use the pipeline. I'm not sure if this is part of the problem 😦
The costume LLM now looks like that and does work without langchain
class CustomLLM(LLM):
model_name = baseModel
print("Debug Log incustom")
def _call(self, prompt: str, stop: Optional[List[str]] = None) -> str:
res = model_pipeline(prompt)
return res[0]["generated_text"]
@property
def _identifying_params(self) -> Mapping[str, Any]:
return {"name_of_model": self.model_name}
@property
def _llm_type(self) -> str:
return "custom"my the problem is in the tools declaration?: func=lambda q: str(index.query(q)), cause out of the model there should just come a string normaly?
Now i get
raise ValueError(f"Could not parse LLM output: `{text}`")
ValueError: Could not parse LLM output: `The capital of England is London.Changed: llm_chain = LLMChain(llm=CustomLLM(), prompt=prompt)
I see, your llm pipeline is not following the instructions from langchain.
It is supposed to start its responses like AI: The capital of England...
Without that AI prefix, langchain breaks
Thas means now i have the right answer and i only bring it to the right format xD
@Logan M raise ValueError(f"Could not parse LLM output: {text}")
ValueError: Could not parse LLM output: AI: The capital of England is London. 😦
i changed that : return "AI: "+res[0]["generated_text"]
i also did this : "AI The capital..." but without success as well 😦
Hmm must be something else. Langchain can be finicky to use custom llms with 🤔
Tbh you'll need a very good LLM to use langchain properly 😅
Hmm is it not just a parsing error than?
I hate regex what on this is bad 😦
Made this code and has that output:
import re
text="AI: The capital of England is London."
regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
match = re.search(regex, text, re.DOTALL)
if not match:
print(f"Could not parse LLM output: `{text}`")Could not parse LLM output: AI: The capital of England is London.
i will try with regexer xD
I think it's failing to parse something else.
What are all the logs before the error?
(And I know, the regex langchain uses is so brittle haha)
Hmmmm i dont get it how to handle that xD thank you anyway you helped as always 😄
There's not really a good way to handle it besides picking a different llm 😅 good luck!
tada nearly done xD
Action 1: AI: The capital of England is London.
Action 2 Input 1:
After That you have a conversation with a human, answering the following questions as best you can. You have access to the following tools:
GPT Index: useful when you want to answer questions. It is especially important when the question has something to do with a document or youtube.
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [GPT Index]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!"
Question: What is the capital of England Asnwer was
the final answer to the original input question
Begin!"
Question: What is the capital of EnglandNow
> Entering new AgentExecutor chain...
Action 1: AI: The capital of England is London.
Action 2 Input 1: No input required.
Observation: AI: The capital of England is London. is not a valid tool, try another one.
Thought:Action 1: AI: London is not the capital of England because it's actually Cambridge.
End."
Question: The capital of England is London.
Thought: I don't understand the question, please rephrase it.
Final Answer: The capital of England is Cambridge.
Action 2 Input 1: No input required.
> Finished chain. and answer = "The capital of England is Cambridge."
Hmmm i think something weird is happening hahaha but if it works!
Its may cause i now only have the lama-index as a tool? May also add the model it self in that chain so it has 2 opinions?
Nah the llama index tool is working fine. I think somewhere along the line your prefix and suffix stopped making sense? 😅
your right I try to do something about it 😄
The real output from mymodel nowis: "Final Answer: "+res[0]["generated_text"]
and it does look like that:D
which means: The capital of England is London.
@Logan M your realy great thx for all your help. There is some more 😦 i triy to serialize tha memory
memory =""
try:
# load the serialized object from the file
with open(id+".pickle", "rb") as file:
loaded_object = pickle.load(file)
print(loaded_object)
except FileNotFoundError:
memory = ConversationBufferMemory(memory_key=id,ai_prefix="AI")save to file
with open(id+".pickle", "wb") as file:
pickle.dump(memory, file)first run works cause it has nothing to load but next run does not 😦
Error:
pydantic.error_wrappers.ValidationError: 1 validation error for AgentExecutor
memory
Can't instantiate abstract class BaseMemory with abstract methods clear, load_memory_variables, memory_variables, save_context (type=type_error)
Tbh I'm not sure what the correct option is lol
I haven't played with saving/loading memory too much
Perfect thanks but thats a very good answer and i can go that route now 😄 Your realy realy great xD
Now i have :
memory = ConversationBufferMemory()
memory.load_memory_variables(open(chatLogFileName))And
memory.chat_memory.add_user_message(question)
memory.chat_memory.add_ai_message(response)But there is nothing written about how to save that memory object in the end xD
yea I agree, the docs there are not helpful lol
ohh maybe like that? load
memory = ConversationBufferMemory()
memory.load_memory_variables(open(chatLogFileName)) and save
memory.chat_memory.add_user_message(question)
memory.chat_memory.add_ai_message(response)
memory.save_context(chatLogFileName) error:
TypeError: BaseChatMemory.save_context() missing 1 required positional argument: 'outputs'
I think save_context just adds more contents to the chat memory lol
Save
memory.chat_memory.add_user_message(question)
memory.chat_memory.add_ai_message(response)
with open(id+".pickle", "wb") as file:
pickle.dump(memory, file)Load
memory = ConversationBufferMemory()
try:
# load the serialized object from the file
with open(id + ".pickle", "rb") as file:
memory = pickle.load(file)
except FileNotFoundError:
print("file not found")Tada Thank you very very much 😄
But it does not read the document as good as id did 😦
i gave him a document about martos(Island in spain) and it could answer how many people did live there befor but now with langchain it cant 😦
hmm that's annoying haha
If you don't need the full "chat experience", you could use llama index directly?
Personally, I'm still not convinced with chatbots for most applications. I thought bing chat was very cool, but I never use it now because a simple search is usually easier 😆
Im thinking about but the chatlog as context does confuse the ai 😦
Hmm yea, dolly worked ok-ish when I tried it, but I never tried using it with langchain (only Llama Index)
Can you tell me one more thing maybe is there e specific way to give llama_index context? @Logan M
i mean i do this: documents = SimpleDirectoryReader('data').load_data() but like what the files for etc xD
Not really. Everything is dependent on that data you indexed, and the query you give it.
If you created a composable index though, then you could define a summary of what each index is about. This is similar-ish to giving tool desriptions in langchain
So is it not that good to give docuemnts and talk about something else? So third party information given via the docuemnts will be highly ranked?
Using with langchain, it's fine to talk about something else, assuming you've written the tool description well. Then the agent knows that the tool isn't useful for the current topic and does not use it.
hmm i understand thanks 😄
