Find answers from the community

Updated 3 months ago

Output format

Are there some general rules / guidelines to ensure a more consistent response from the the API when making a query in terms of the formatting of the response? I am using some TOKEN_START and END and asking it fill in the middle as part of the QA_TEMPLATE and such, but the response format is not always 100%. I am currently trying to catch these edge cases and build code around them, but maybe I am making it too hard on myself so I figured I ask here!

7 comments

LLogan M

If you have some guidelines or rules, you could try customizing the text_qa_template and the refinr_template to explicitly state those rules 👀

mmcmancsu

Yeah, that is the plan for now, just curious if there is something that forces a better fit into that template.

LLogan M

Are you trying to output json or a similar structured output?

mmcmancsu

currently json yes, but also contemplating the streaming capability, which complicates the 'after the fact parsing'

LLogan M

Hmm, yea prompt engineering might be the best way, especially if you want to move to streaming.

There are options like gaurdrails, but this adds latency and extra LLM calls 😅

mmcmancsu

Any pointers are welcomed, but otherwise I'll keep poking around! Appreciate the response

mmcmancsu

Semi-answering my question. I found some good details just googling around for "Prompt Engineering" - youtube, blogs, etc.

Also, found this: https://prmpts.ai/ to help design it out!

Add a reply