Find answers from the community

Updated 2 months ago

Output format

Are there some general rules / guidelines to ensure a more consistent response from the the API when making a query in terms of the formatting of the response? I am using some TOKEN_START and END and asking it fill in the middle as part of the QA_TEMPLATE and such, but the response format is not always 100%. I am currently trying to catch these edge cases and build code around them, but maybe I am making it too hard on myself so I figured I ask here!
L
m
7 comments
If you have some guidelines or rules, you could try customizing the text_qa_template and the refinr_template to explicitly state those rules πŸ‘€
Yeah, that is the plan for now, just curious if there is something that forces a better fit into that template.
Are you trying to output json or a similar structured output?
currently json yes, but also contemplating the streaming capability, which complicates the 'after the fact parsing'
Hmm, yea prompt engineering might be the best way, especially if you want to move to streaming.

There are options like gaurdrails, but this adds latency and extra LLM calls πŸ˜…
Any pointers are welcomed, but otherwise I'll keep poking around! Appreciate the response
Semi-answering my question. I found some good details just googling around for "Prompt Engineering" - youtube, blogs, etc.

Also, found this: https://prmpts.ai/ to help design it out!
Add a reply
Sign up and join the conversation on Discord