Few shot

Super cool! I wonder if this could work with our agent tool spec stuff too 🤔

Yeah, was thinking about that too! Can try sketching something out.😁

Nice one was actually thinking how to do few shot with function api yesterday

Was looking at this the other evening and came across this thread here:

https://community.openai.com/t/few-shot-and-function-calling/265908/15

basically, for doing few-shot with function-calling examples, we spoof prior prompt -> function call pairs in the chat history. doesn't seem too bad

what's clear though is that we'd need to write an abstraction layer for the few-shot decorator. the current implementation is just the "function docstring is a prompt, your examples go in prompt"

so basically, if we wrap an OpenAIPydanticProgram object, the __call__ implementation routes to a new piece of logic, that instead of modifying a function docstring, modifies the initial chat history that goes in to the openAI call

this is all kinda fine because the whole "implicit" function that's just generating something that fits the description/examples is very much a fire-and-forget operation that has no state, i.e. you can afford to just pad the chat history with a few examples that will make the chance that you get a good answer higher

a bit harder when thinking about the tool spec, since function tool calls happen in the middle of a conversation where prior context is important, and they can chain into each other

so maybe putting the examples in the function description is actually better/less messy for that use-case. but that feels a bit janky for some reason.

I think putting examples in the docstring makes sense!

We actually do that for some of the tools on llama-hub actually 👀

cool, know which ones off the top of your head? will take a look how it's formatted etc

I guess it's not as common as I thought. Most docstrings just define args and return values.

These two have few-shot-ish though
https://github.com/emptycrown/llama-hub/blob/main/llama_hub/tools/graphql/base.py
https://github.com/emptycrown/llama-hub/blob/main/llama_hub/tools/shopify/base.py


These ones aren't few-shot, but are interesting lol
https://github.com/emptycrown/llama-hub/blob/main/llama_hub/tools/chatgpt_plugin/base.py#L7
https://github.com/emptycrown/llama-hub/blob/main/llama_hub/tools/code_interpreter/base.py

For the first two I think the standard decorator should work as is. although interestingly, neither provide the outputs. Which means I should probably update the package to support input-only examples, right now it really expects input/output pairs 😄