I want to generate a list of questions

LLeMoussel

I want to generate a list of questions in French.
To generate the questions, I use data_generator.generate_questions_from_nodes()
This API generates the following prompt:

Plain Text

Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge.
generate only questions based on the below query.
{query_str}

Is it possible to define a custom prompt for question generation? How can I do this?

Ref: https://gpt-index.readthedocs.io/en/latest/examples/evaluation/QuestionGeneration.html

6 comments

LLeMoussel

I made it this way:

Plain Text

from llama_index.prompts import PromptTemplate

text_question_template_str = (
  "Les informations contextuelles sont ci-dessous.\n"
  "-----------------------------------------------\n"
  "{context_str}\n"
  "-----------------------------------------------\n"
  "Compte tenu des informations contextuelles et non des connaissances préalables.\n"
  "Générer uniquement des questions basées sur la requête ci-dessous.\n"
  "{query_str}"
)
text_question_template = PromptTemplate(text_question_template_str)
data_generator.update_prompts({"text_question_template": text_question_template})

Is this a good solution?

LLeMoussel

Indeed the content of query_str is always in English.

Plain Text

query_str: 
"You are a Teacher/Professor. Your task is to setup 10 questions for an upcoming quiz/examination. The questions should be diverse in nature across the document. Restrict the questions to the context information provided."

How to change query_str to French?

Attachment

LLeMoussel

Here is the solution

LLeMoussel

Plain Text

from llama_index.prompts import PromptTemplate
from llama_index.evaluation import DatasetGenerator

# https://github.com/run-llama/llama_index/blob/main/llama_index/evaluation/dataset_generation.py#L97
data_generator = DatasetGenerator.from_documents(
  documents,
  num_questions_per_chunk = 10,
  text_question_template = PromptTemplate((
  "Les informations contextuelles sont ci-dessous.\n"
  "-----------------------------------------------\n"
  "{context_str}\n"
  "-----------------------------------------------\n"
  "Compte tenu des informations contextuelles et non des connaissances préalables.\n"
  "Générer uniquement des questions basées sur la requête ci-dessous.\n"
  "{query_str}"
  )), 
  text_qa_template = PromptTemplate((
  "Les informations contextuelles sont ci-dessous.\n"
  "-----------------------------------------------\n"
  "{context_str}\n"
  "-----------------------------------------------\n"  
  "Compte tenu des informations contextuelles et non des connaissances préalables, réponder à la requête.\n" 
  "Requête: {query_str}\n"
  "Réponse:"  
  )),
  question_gen_query = (
      "Vous êtes un enseignant/professeur."
      "Votre tâche consiste à créer {num_questions_per_chunk} questions pour un quiz/examen à venir."
      "Les questions doivent être de nature diverse dans tout le document."
      "Limiter les questions aux informations contextuelles fournies."
  )
)

rravitheja

cool. Hope it worked as expected.

LLeMoussel

Yep ! That works well. Now, I still have to find/test LLM models for French.

Add a reply

Find answers from the community

I want to generate a list of questions