Find answers from the community

Updated last year

Logan M hi i tried to ask kapa ai but

At a glance

@Logan M hi, i tried to ask kapa.ai but it doesnt give me convincing answer. When i am using tree_summarize with list_index it sends many requests to openai and sometimes it stucks, or even gets timeout. What can i do with this? Split my file into chunks? Or implement timeout, if there is no answer within 5 minutes stop making queries and return what is already done. Is it possible? What would you recommend to do?

3 comments

LLogan M

The list index sends every node in your index to openai, and builds a bottom of tree of answers

So it's already split into chunks. Maybe your index is too big? Do you actually need to send every node to the LLM?

You can also try setting

Plain Text

# apply nesting if in a notebook or server
import nest_asyncio
nest_asyncio.apply()

index.as_query_engine(response_mode="tree_summarize", use_async=True)

mmaxanjo512

i tried to use_async. Maybe my file is too big. I have reviews that i want to analyze, when i have 200 reviews and i make an index with it, everything is fine. When i have 400, it stucks. I want to figure it out how can i make this work, i have no idea.

LLogan M

What are you trying to do? Summarize 400 reviews?

Add a reply