What s the success rate on getting a LLM

My personal experience (based on 1 project that went into "production" but then had <10 daily active users)

~80% if you take the raw response. But it was 100% if you strip everything before the first { or [ and strip everything after the last } or ].

The JSON was always valid, and the only failure mode was from the model saying stuff like Sure. Here is your answer: before the JSON

You may be familiar with https://js.langchain.com/docs/modules/model_io/output_parsers/

I have never tried it or even looked at it closely. I blindly guesses it's no better than the 3 line function we had that stripped text surrounding the JSON.

Sweet, thanks. as long as it's close to 100%, perfect. Trying to iron out things on the TS side right now