So uh, what setting field should we use to let openai know the max completion tokens we want? I just tried setting max_tokens in the llm but, it just... stopped mid-sentence? 🤔 is that normal? or is there another setting?
sounds like you set max_tokens and you set it too short?
This is pretty expected. max_tokens is just a hard cap, it doesn't know if the model is done talking or not.
If you want to try and instruct the actually length, things like specifying the number of sentences you want in the prompt is probably a good starting point. It can be a little difficult though
