Find answers from the community

Updated 2 years ago

Run on each node

example:

[for this node?] using the context provided by the index summary [previously calculated], identify all the distinct narrative scenes in this passage, along with all the characters who participate in each scene.
1
L
w
A
8 comments
This is another feature on my todo list πŸ˜… basically the idea is you set response_mode="accumulate" and it will just run your query on every node, returning a list of responses
yes, that would be a super useful feature. I suppose the alternative would be to simply do all these queries outside llama, but then I would have two sets of service contexts & code to manage.
Yup for sure, totally understand! I'm hoping to get this idea implemented in the next week or two πŸ™ but also open to someone else trying a PR if they have the time! πŸ’ͺ
where would I start?
If you are up for it, it would probably be in here: https://github.com/jerryjliu/llama_index/blob/main/gpt_index/indices/response/builder.py#L378

Rather than switching to the refine prompt, is response_mode="accumulate", the code would keep track of each response... somehow. Maybe a list. Maybe a giant response string.
@kapa.ai How can i make accumulate response mode return a summary of all the responses ?
Let me check the knowledge base for you. One moment! :books:
----------------------
The accumulate response mode in LlamaIndex applies a query to each text chunk and accumulates the responses into an array. It then returns a concatenated string of all responses. This is useful when you need to run the same query separately against each text chunk. However, it does not inherently provide a summary of all the responses.

To get a summary of all the responses, you might need to use a summarization tool or technique on the accumulated responses. Unfortunately, the provided extracts do not provide information on how to do this directly within LlamaIndex.

Here is how you can use the accumulate response mode:

Plain Text
from llama_index.response_synthesizers import Accumulate
from llama_index import ServiceContext

# Initialize the service context and the Accumulate response synthesizer
service_context = ServiceContext.from_defaults(llm=llm)
summarizer = Accumulate(service_context=service_context)

# Get the response
response = summarizer.get_response("your_query", text_chunks)


In this code, replace "your_query" with your actual query and text_chunks with your actual text chunks.

For more details, you can refer to the LlamaIndex documentation ().
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
Add a reply
Sign up and join the conversation on Discord