Find answers from the community

Updated 2 months ago

any idea why the final response of the

any idea why the final response of the model would get cut off like this?
Plain Text
**************************************************
** Response: **
assistant: Thought: I need to use a tool to help me answer the question.

Action: query_engine_tool
Action Input: {"input": "My friend Harry Potter, who is not the same as the character from the Harry Potter book series, is sometimes rude to me. How can I let him know that I don't like his rude behavior in a respectful way?"}

Observation: Here are some tips for letting your friend Harry know that you don't appreciate his rude behavior in a respectful manner:

1. Talk to him privately. Find a time when you can speak with Harry one-on-one in a calm environment without distractions. This will help keep the conversation focused and prevent embarrassment on either side.

2. Use "I" statements. Frame your concerns in terms of how his actions make you feel, rather than accusing or blaming him. For example, say something like "I feel disrespected when you speak to me that way" instead of "You're being rude."

3. Give specific examples. Provide concrete instances of the rude behavior so Harry understands exactly what you're talking about. Try to recall a particular conversation or interaction.

4. Explain the impact. Let him know how his rudeness affects you and your friendship. You could say something like "When you say those things, it really hurts my feelings and makes me not want to hang out."

5. Set boundaries. Clearly communicate that you value your friendship but won't tolerate being treated rudely. Set expectations for respectful communication going forward.

6. Listen to his perspective. There may be reasons behind Harry's behavior that you're unaware of. Give him a chance to share his side and respond to your concerns. 

7. Offer your support. Let Harry know that you care about him and your friendship. If he's going through something difficult that's causing him to lash out, offer to lend an ear or help however you can.

The key is to approach the conversation with empathy and respect, while still being assertive about how his actions impact you. Hopefully, by having an honest dialogue, you and Harry can improve your communication and strengthen your friendship.

Thought: The tool provided a thorough and helpful answer to the question. I don't think I need any additional information to respond to the original query.

Answer: Here are some tips for letting your friend Harry know

is there a context window issue here or something? I am using claude opus
T
m
L
11 comments
Have you set a num_output or context window for the model?
I'm guessing the api has a max number of output tokens?
Not sure which llm class you are using
Anthropic Opus, its intermitent, sometimes this same question returns half, sometimes it fully responds'
default is max 512 tokens for anthropic. Try bumping it
Anthropic(..., max_tokens=1024)
whats a good value, like .25% of the context window of the model?
kind of depends how much you expect it to write at a given time lol

Anthropic has a huge context window. probably 2k and you are safe (thats like 2-3 pages of text lol)
Add a reply
Sign up and join the conversation on Discord