just change llama-index==0.10.15 to llama-index==0.9.15 will fix, but I want to use the new version

I'm upgrading my llama index version from 0.9.32 to 0.10.7, after that I found my python project has a runtime error:




here qa.py and index_server.py are my own python files, but the error comes from llama_index/core/indices/postprocessor.py. can anyone tell me what should I do? thanks in advance!

I want to build a chatbot which can understand and answer user's questions.

there are 2 kinds of questions:

1. pre-set standard questions. these are important, carefully selected, frequently asked questions in the past. we have carefully prepared standard answers to these questions(in the form of videos, articles, etc). when a user's question comes, the chatbot is expected to return the matched standard question(a following module will fetch the standard answer to the question and return to user). the chatbot should never try to answer a matched standard question by itself.
2. other questions. if user's question doesn't match any of the standard questions, then the chatbot should fallback and try to answer it through gpt3.5.

I'm trying to customize a llama index chat engine to fulfil this. the chat engine consists of a query engine(which matches a query against standard question library), a llm(gpt3.5), and a system prompt. it can evaluate whether a question is a standard question or not, which is good. but the problem is, it will continue to forward the question to llm and return the final answer, which costs extra time and the result is useless. how can I stop this and let it just return the matched standard question when a match is found? attached is the code snippet(the full code is in https://github.com/pengxiaoo/llama-index-fastapi/blob/main/app/llama_index_server/index_server.py)

I'm using llama index to build a question answering bot based on my private knowledge base(a bunch of prepared question-answer pairs in csv file). the knowledge base is embedded into llama index's vectorStore. for now everything works well, except the latencies due to LLM api calls. I want to improve in this way: when user asks questions, I want the bot search the vectorStore first, if there is a good match, return directly the matched answer without turn to LLM; if there isn't a good match, then turn to LLM for an answer. the aim is to reduce unnecessary calls to LLM. does anyone knows how to do this? thank a ton!

sometimes the following line of code takes 10+ seconds to get result. how can I set timeout to this call? I want to stop requesting if the response can not be returned in a given time period. but I didn't find related documentation how to do it.

I also have a few concerns about the concurrency. I have an app with about 200k active users in a day, about 30k online users in an hour, 1.5k online users in a minute. I want to give users some AI related functionalities backed by llama index+openai api. assume 10% active users will use. then it would be 150 calls in a min, 3k calls in an hour, 20k calls in a day, to llama index. is this even practical? what level of hardwares will be needed? thanks in advance for any suggestions!

I have a legacy java springboot project, want to integrate llama index or langchain to it. is there java api available for llama index?