forcing prompt

At a glance

A community member is trying to build a chatbot using Llama Index, and the information they want the chatbot to ingest is in a JSON file in Spanish. They want to force the chatbot's responses to be in a single language (Spanish) without any translation, but they have tried writing a system prompt to force the model to answer in Spanish and it didn't work. The community members discuss different approaches, such as changing the QA and retrieval prompt templates, adding examples of good (Spanish) and bad (English) responses, and trying "negative English and positive Spanish" in the prompt, but none of these solutions seem to work. There is no explicitly marked answer in the comments.

AAlfredo Delgado

Hi everyone. I have a noob question if I may post it here. For some context, I'm trying to build a chatbot using Llama Index. The information I want it to ingest for answering is in a JSON file in Spanish, how can I force the agent's responses (I'm using ChatGPT) to be in a single language without any translation? I tried writing a system prompt forcing the model to answer in Spanish but it didn't work. Could you help me? Thanks.

By the way. Awesome stuff Llama Index. Kudos to all developers working on this project.

12 comments

EEmanuel Ferreira

I had this same problem but to return response in JSON, tried different prompts and work, but take a tine, really weird

HHongyi Shi

@Alfredo Delgado did you change the qa and retrieval prompt templates being used? They are in English by default if you translate it into Spanish it might help. Also adding examples of good (Spanish) and bad (English) in your prompt might help.

AAlfredo Delgado

@Hongyi Shi That's a really good idea. Unfortunately I tried that and it didn't work. Look:

Plain Text

TEMPLATE_STR = (
    "Se presenta a continuación el siguiente contexto:\n"
    "---------------------\n"
    "{context_str}"
    "\n---------------------\n"
    "Dada esta información, por favor responde la siguiente pregunta exclusivamente en español: {query_str}\n"
)
QA_TEMPLATE = Prompt(TEMPLATE_STR)

tools = [
    IndexToolConfig(
        query_engine=index.as_query_engine(text_qa_template=QA_TEMPLATE),
        name = "FAQ",
        description="Primera fuente de consulta. Base de datos de preguntas y respuestas frecuentes de la compañía",
        tool_kwargs={"return_direct": True}
    ),
]
toolkit = LlamaToolkit(index_configs=tools)

Yeah, the QA template is exactly a direct translation to Spanish of the English default, but adding that it should answer only in Spanish and it didn't work. The rest is just text that I don't think it's relevant to the issue at hand.

Any other ideas?

Thanks @Hongyi Shi

HHongyi Shi

Did you try the negative English and positive Spanish in your prompt?

AAlfredo Delgado

I'm sorry @Hongyi Shi but I didn't quite understand what you mean with negative English and positive Spanish. Do you mean writing a prompt telling it to not translate to English and answer everything in Spanish?

AAlfredo Delgado

Or giving it examples of how not to answer?

AAlfredo Delgado

(i,e. not English)

AAlfredo Delgado

Well, I wrote a negative English and positive Spanish example and the situation is the same: english still. 😢

AAlfredo Delgado

Thanks again @Hongyi Shi

HHongyi Shi

Ah hmm thats too bad

HHongyi Shi

If you want to dig deeper you can set logger level to debug to see what is actually being sent to the llm

HHongyi Shi

I think part of the prompt could still be in English

Add a reply

Find answers from the community

forcing prompt