Is there any sort of leaderboard which

Not that I know of, But I can give a short answer -- none of them 🙂 At least out-of-the-box

turns out agents are hard

lol ok that works too. I've been using GPT-4 but was considering trying some open source options but didn't want to shoot in the dark if there were any good candidates

At least with 7b-13b models, all I tried were pretty unreliable. Hallucinating the react agent loop, giving tool inputs in the wrong format, etc.

Pretty tricky

yeah i tried the react agent with Gemini and had trouble with conversational prompts. It kept insisting on always using a tool

yeaaa thats gemini 😅

gpt-3.5 is ok-ish if you need lower cost

Thanks you saved me some time