Find answers from the community

J
JW
Offline, last seen 2 weeks ago
Joined September 25, 2024
I found that RetrieverQueryEngine does not support "custom_query_template". I think the document is wrong. However I found a way to add my own template by passing it to response_synthesizer

# configure response synthesizer response_synthesizer = get_response_synthesizer(text_qa_template=custom_query_template)
2 comments
J
L
Hi, I'd like to process the documents ( lemmatization and removing stopwords) before creating index. And when query, I want use the processed text to find related resources but use original question when sending to gpt-3.5. Thess are my steps:

  1. Process documents and generate vector store index
  2. Process the question
  3. Find top k related sources using processed question
  4. Send original question and processed sources context to GPT to answer the question
Is it possible to do this in llama index ? (maybe using QuestionAnswerPrompt template ?)
6 comments
J
L
How do I get sources used when integrating with agents? I know this feature was added recently but I cannot find the documentation for this?
4 comments
L
J
@kapa.ai
Plain Text
# can define filters specific to this vector index (so you can
# reuse pinecone indexes)
metadata_filters = {"title": "paul_graham_essay"}

How does this step work when querying on GPTPineconeIndex
2 comments
k
@kapa.ai I wonder if GPTPineconeIndex works better than GPTSimpleVectorIndex for large file such as a 10K SEC file?
2 comments
k
@kapa.ai Hi, How can I set chunk_size and overlap size when using GPTPineconeIndex and how can I store the built index for future use?
9 comments
J
L
k
J
JW
·

Graph query

I add there
Plain Text
{
            "index_struct_type": "simple_dict",
            "query_mode": "default",
            "query_kwargs": {
                "similarity_top_k": 3,
            },
        }

But it seems the query use about 8 nodes.
18 comments
J
L
J
JW
·

```

Plain Text
    graph = ComposableGraph.from_indices(
        GPTListIndex,
        index_arr,
        index_summaries=summaries,
        service_context=service_context
    )

Hi when I add service_context to ComposableGraph.from_indices I always get this error ValueError: Got a larger chunk overlap (20) than chunk size (-42), should be smaller.

This is my service_context:
Plain Text
# set maximum input size
max_input_size = 256
# set number of output tokens
num_outputs = 256
# set maximum chunk overlap
max_chunk_overlap = 20

chunk_size_limit = 512
# define LLM
llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.2, model_name="gpt-3.5-turbo", max_tokens=num_outputs))
prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper, chunk_size_limit=chunk_size_limit)
9 comments
j
L
J
J
JW
·

Chunk size

When I make index, I set chunk_size_limit = 512. However, I see one node text is very long and its token is about 3000. Would it be at most 512 tokens ?
3 comments
J
L
J
JW
·

Refine template

Plain Text
# Refine Prompt
CHAT_REFINE_PROMPT_TMPL_MSGS = [
    HumanMessagePromptTemplate.from_template("{query_str}"),
    AIMessagePromptTemplate.from_template("{existing_answer}"),
    HumanMessagePromptTemplate.from_template(
        "We have the opportunity to refine the above answer "
        "(only if needed) with some more context below.\n"
        "------------\n"
        "{context_msg}\n"
        "------------\n"
        "Given the new context, refine the original answer to better "
        "answer the question. "
        "If the context isn't useful, output the original answer again.",
    ),
]

this is my refine template
Plain Text
# define query configs for graph 
    query_configs = [
        {
            "index_struct_type": "simple_dict",
            "query_mode": "default",
            "query_kwargs": {
                "similarity_top_k": 1,
                "include_summary": True,
                "refine_template": CHAT_REFINE_PROMPT_TMPL_MSGS
            },
            "query_transform": decompose_transform
        },
        {
            "index_struct_type": "list",
            "query_mode": "default",
            "query_kwargs": {
                "response_mode": "tree_summarize",
                "verbose": True,
                "refine_template": CHAT_REFINE_PROMPT_TMPL_MSGS
            }
        },
    ]

this is my query_configs

However, when I run response = graph.query(q, query_configs=query_configs), I got error
Plain Text
line 117, in refine_response_single
    refine_template = self.refine_template.partial_format(
AttributeError: 'list' object has no attribute 'partial_format'
2 comments
J
L
J
JW
·

```

Plain Text
from llama_index.langchain_helpers.agents import LlamaToolkit, create_llama_chat_agent, IndexToolConfig, GraphToolConfig
agent_chain = create_agent(toolkit, llm)
    while True: 
        query = input("What do you want to ask? ")
        print('question: ', query)
        response = agent_chain.run(input=query)
        print(f'Agent: {response}')

When integrating a language model with an agent_chain, is it possible to obtain the resources or chunks used as the context by the agent_chain?
32 comments
L
J
Hi, I create an agent_chain using create_llama_chat_agent. When I ask the same question about 10 SEC file, sometimes it provide me an expected answer, but some time it gives me The original answer remains relevant and does not require refinement based on the new context provided. Why this happened ? how can i fix this ?

Can anyone help me?

Thanks
2 comments
J
L