Then another issue is that I'm trying to

Then another issue is that I'm trying to run it on a compact LLM such as Phi3 (small one).

However, even though the relevant node are well retrieved, after the LLM aren't capable to exploit well the data retrieved.

Plain Text

    instruction_str = (
        "1. Convert the query to executable Python code using Pandas.\n"
        "2. The code should represent a solution to the query.\n"
        "3. PRINT ONLY THE EXPRESSION.\n"
    )

    pandas_prompt_str = (
        "You are working with a pandas dataframe in Python.\n"
        "The name of the dataframe is `df`.\n"
        "Here is a list of columns from the dataframe `df` with explanations for each column.\n"
        "{columns_explain}\n\n"
        "Here's are some relevant example rows (values in the same order as columns above)\n"
        "{relevant_node}\n\n"
        "Follow these instructions:\n"
        "{instruction_str}\n"
        "Query: {query_str}\n\n"
    )

Is there a way to force him better ?

2 comments

WWhiteFang_Jr

Small LLMs are not that good at following instructions. you can try going with bigger llms. Try with ollama, they have quantized llama3 that you can try in colab I think

AAlex

I tried with Llama3 (using Ollama), however, he still generates like answer that cannot be "eval()" : even though I did ask him (not here, I did test before). I will try again -> Not sure to have tested it when the good retrieval was done.

For instance: one answer he can give for the pandas instructions is:

Plain Text

Here's one example of a pandas instruction to retrieve "something"
\`\`\`python
#Some python code
\`\`\`

Add a reply

Find answers from the community

Then another issue is that I'm trying to