What are some strategies to limit hallucinated responses when using structured output parsers with Llama models? For example, I ask the model to identify the number of hours that employees work per week from paragraph text. My pydantic class looks like this:
python class WorkWeek(BaseModel):
hours_per_week: Optional[float]
employee_type: Optional[str]
I find I
always get some answer back like
hours_of_work = 40
and
employee_type = 'full-time'
regardless of whether the document contains relevant text or not. I could create a document that just has the text
This is a test
and I would still get results back that are clearly not from the document like shown above. Interestingly, when I simply ask the model for a basic freeform text response, I get an answer that makes sense when the document doesn't mention hours of work (something like
Based on the context, there isn't enough information to answer your question
).