response = await query_engine.aquery("What is the business plan of ?")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: 'await' outside function i am getting this error
Hmm why is the notebook using aquaery lol
i remove await get another error
<coroutine object BaseQueryEngine.aquery at 0x15fd42890>
sys:1: RuntimeWarning: coroutine 'BaseQueryEngine.aquery' was never awaited
can help me fix the aquery. i am new to llama index
i changed aquery to query then get another error : openai.error.APIConnectionError: Error communicating with OpenAI
I saw someone else with this error above, maybe an issue with openai servers? 🤔 just woke up haha I can check back in a bit
ok please let me know as i am testing now
The notebook sets use_async=False, but really you probably want this to be True to make the response faster.
Basically it's generating a set of sub questions for a query, and then it gathers answers and returns a final response using the answers to all sub-questions
Can show me your final code
query_engine = SubQuestionQueryEngine.from_defaults(
query_engine_tools=query_engine_tools,
service_context=service_context,
use_async=True,
)
response = query_engine.query(
"How was Paul Grahams life different before and after YC?"
)Same issue for multi step query engine
Can help check if the read the docs code is correct or got bugs?
I can try running it in a bit, but again, the sub question engine like above should work better than the multistep thing 🤔
My use case is getting out detailed information and examples from a book regarding a topic
Which query engine would you recommend?
Sub question engine is more maintained haha 😄
raise error.APIConnectionError("Error communicating with OpenAI") from e
openai.error.APIConnectionError: Error communicating with OpenAIi am getting the same error @Logan M
funny thing is it generaetes a subquestion but then gives openaI error as above ^
how can i make sure more details and subqustions are generated? for now i am getting a very short answer
Maybe include you want more details in your query?
i have a differenton enow
vector_store = SupabaseVectorStore(
postgres_connection_string=connectionString,
collection_name="llama_demo",
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
# NOTE: use the below during init
# index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
llama_debug = LlamaDebugHandler(print_trace_on_end=True)
callback_manager = CallbackManager([llama_debug])
# llm = OpenAI(model="gpt-3.5-turbo-16k",temperature=0)
llm = ChatOpenAI(model="gpt-3.5-turbo-16k",temperature=0)
llm_predictor = LLMPredictor(llm=llm)
# llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo-16k"))
# llm = ChatOpenAI(temperature=0, model="gpt-4")
index= VectorStoreIndex.from_vector_store(vector_store = vector_store, storage_context=storage_context)
query_engine =index.as_query_engine()
query_engine_tools = [
QueryEngineTool(
query_engine=query_engine,
metadata=ToolMetadata(
name="vector index", description="The entire book"
),
)
]
service_context = ServiceContext.from_defaults(callback_manager=callback_manager, llm_predictor=llm_predictor)
sub_query_engine = SubQuestionQueryEngine.from_defaults(
query_engine_tools=query_engine_tools,
service_context=service_context,
use_async=True,
)
# query index
# query_engine = index.as_query_engine(response_mode="tree_summarize")
query1 = "what is the thesis"
response = sub_query_engine.query(query1)
print(response)getting the below error for the above code
ns/3.11/lib/python3.11/site-packages/llama_index/output_parsers/utils.py", line 12, in parse_json_markdown
raise OutputParserException(
llama_index.output_parsers.base.OutputParserException: Got invalid return object. Expected markdown code snippet with JSON object, but got:
[That happens sometimes when the llm doesn't structure its response properly. The sub questions have to be structured a certain way so we can parse them
Was actually going to make a change in a bit to make it less strict.
Btw tho, if you use gpt-3.5 or gpt-4, it should switch to a more reliable mode (at least in the latest version)
notedthanks. p.s. i am always getting this error
yes gpt3.5 already in this.
btw i am planning to use llama-index in my production app. will the udpate fix it fundamentally or still depend on the non deterministic output of the llm?
Already 3.5? Maybe try installing from source to get the more reliable thing I was talking about, can't remember if it's on pypi yet
pip install --upgrade git+https://github.com/jerryjliu/llama_index
The more reliable version I mention is using the function calling api. I think there's quite a bit less risk of issues there. Only problem could be if the LLM hallucinates the name of an query engine tool 🤔
if i do this i dont need to uninstall old version right
Yea, the --upgrade option should take care of the old version
same issue after upgrading. i think pypi is same version
pypi will be the same version, but now it will be the latest from git
Same exact issue? Are you sure you are using gpt-3.5? Because it shouldn't be using this part of the code 🤔
Can I see how you setup the llm?
Using llm and llm predictor here. Can you check this block
Slightly incorrect I think, you are using langchain's LLM class. Here's an adjusted sample
from llama_index.llms import OpenAI
llm = OpenAI(model="gpt-3.5-turbo-16k", temperature=0)
service_context = ServiceContext.from_defaults(llm=llm, callback_manager=callback_manager)
# NOTE: use service context here
index= VectorStoreIndex.from_vector_store(vector_store = vector_store, service_context=service_context)
# NOTE: continue to also use service context here
sub_query_engine = SubQuestionQueryEngine.from_defaults(
query_engine_tools=query_engine_tools,
service_context=service_context,
use_async=True,
)think i got it from one of the docs pages
yep this is the incorrect doc
laaaame, will update that
thanks for your helps appreciate
but have to feedback the docs are all over the place
if you werent there next to impossible fo rme to make it work haha
because docs are incorrect
i am trying to get it to production, hope you can support us. btw we are fyva.ai based in london. our founder knows jeremy
so how can i make my question into multiple subquestions to get a detailed answer
currently the answer is very short
Can you give more details on what you expect? You may have to modify the query a bit to get it to look into things more
so my use case is querying an academic text /long form . lot of pages
i need to get detailed summarization based on the prompt given
for this i have to make sure the correct data is retrieved
.e.g if its a business plan = then if i ask please detailed about their marketing plan , it should retrieve the right data from my vector database and then run the prompt. so wondering if sub question query engine or multi step query engine can help me with this
i tried sub question query engine as per your advise, but it is not generating enough details. how can i get it to generate more sub prompts?
@Logan M any luck for update
The only thing I can think of for getting more details is just asking for more in the query 😅
Also note that by default the model is limited to 256 tokens of output. If you start seeing things get cut off you have to increase max_tokens in the LLM definition
i thought max_tokens is infinity by default based on openai documentation
i am not getting token limit issue
