That's an azure issue sadly 😅 Will have to go to them to fix that, I have no idea
Ok, did you know about this issue ?
also, why I can't put an llm predictor in addition of llm in service context ?
I've seen it on and off before, they seem to have a very strict filtering policy
llm_predictor is a wrapper around an LLM
When you put in just an LLM, we wrap it with LLMPredictor for you
So, they are the same thing really, both cannot be specified, only one
ok, I was using this to add a query for always answering in french, but it does not work with llm predictor also, how to proceed ?
btw, finally , using directly the summary index/the vector store index, it
gives good answers compared to using the QASummaryQueryEngineBuilder
Like when I ask for names, the first one answer properly, whereas the second one tells me he does not have the answer
what is different ?
is the node system breaking all ?
Your going to have to give me more details on this one. Like sample code to reproduce the issue you are having, it's a little unclear
Same for this, not entirely sure what you mean here 😅
llm = AzureOpenAI(
model="gpt-35-turbo-16k",
engine="gpt-35-turbo-16k",
api_key=api_key,
api_base=api_base,
api_type=api_type,
api_version=api_version,
)
# You need to deploy your own embedding model as well as your own chat completion model
embed_model = OpenAIEmbedding(
model="text-embedding-ada-002",
deployment_name="text-embedding-ada-002",
api_key=api_key,
api_base=api_base,
api_type=api_type,
api_version=api_version,
)
prompt_helper = PromptHelper(context_window=16384, num_output=2048)
service_context = ServiceContext.from_defaults(
llm=llm,
prompt_helper=prompt_helper,
embed_model=embed_model,
)
set_global_service_context(service_context)
summary_text=(
"Des informations contextuelles provenant de plusieurs sources sont présentées ci-dessous.\n"
"---------------------\n"
"{context_str}\n"
"---------------------\n"
"Étant donné les informations provenant de sources multiples et sans connaissances préalables,"
"répondre à la requête.\n"
"Requête : {query_str}\n"
"Réponse : ")
qa_text=("L'information sur le contexte est ci-dessous.\n"
"---------------------\n"
"{context_str}"
"---------------------\n"
"Compte tenu des informations contextuelles et non des connaissances préalables,"
"répondre à la requête.\n"
"Requête : {query_str}\n"
"Réponse : ")
qa_template1 = PromptTemplate(summary_text)
qa_template2 = PromptTemplate(qa_text)
summary_index = SummaryIndex(
documents,
service_context=service_context,
)
vector_index = VectorStoreIndex(
documents,
service_context=service_context,
qa_text=qa_text
)
vector_query_engine = vector_index.as_query_engine()
list_query_engine = summary_index.as_query_engine(response_mode="tree_summarize")
#query_engine=index1.as_query_engine()
response = list_query_engine.query("Make me a resume of the situation, detailing the main points.")
print(response)which shows basically what I aim to do
so, here, the query works fine with azure
using azure directly and langchain, it works too, even in french
using llama_index, and french, it shows me the filter warning
using llama_index, and openai, and the QASummaryQueryEngineBuilder (which choose bnetween two query engine + other things I did not get)
it worked fine with openai, but not now with azureopenai
it seems to use a additional node parser which was not visible with the two classic index, but I don't really know
but it seems do exerce a conflict with azureopenai too
right, but you aren't querying the same things with azure and langchain as you are with llama-index. If I had to guess, something about the summary that the LLM is writing is causing it to get filtered. You can try setting debug logs to inspect the requests, but I can't really help beyond that for this issue
import openai
openai.log = "debug"
So you are using QASummaryQueryEngineBuilder, and you are saying it doesn't work. But what doesn't work exactly? Can you give an example? (I think you might have already, but just want to make sure I'm on the same page)
like with openai it gives good results
with azure, not at all, indeed, it can't answer question whereas the index alone can (both summary and vectorstore)
You'll see there is a description for the two "tools"
this is what the LLM reads when deciding which one to use
So you can write them yourself to be more descriptive or explicit (or even write them in french)
For instance here are two results, one from the Engine Builder, and the second from the vectorstore querying
The car plate is not mentioned in the given context information.
Yea I believe you -- just pointing you in the direction of how to optimize this
The documents and the llm/embeddings are the same
yeah np, just wanted to know why knowing it seems to only be a router
but as I saw, this router also use nodes parser ?
(in the class of the engine builder)
is this could change anything ?
everything uses a node parser when you buld an index -- I don't think that's related to any issues here though
Yeah, so i really don't know lol
Also btw I was just wondering if the treeindex is more/less or as efficient as the summary index for summary purposes of multiple documents but all loaded with the directory loader
Tree index takes much longer to build, but queries will be faster. So it could work better
Yea try the link above I sent and play around with the descriptions of each item. Writing them in french may help
okey thanks, indeed the main point is to dinamically switch between indexes
for the moment I was looking for this, but does not work for my ase it seems:
InvalidRequestError: Resource not found I got this error now
After initializing the query engines, I've followed the way of the doc, but it shows me this error
since you are using azure, you'll need to pass the service context into each sub-index and the router
Here's an improved example
vector_index = VectorStoreIndex.from_documents(..., service_context=service_context)
summary_index = SummaryIndex.from_documents(..., service_context=service_context)
...
query_engine = RouterQueryEngine.from_defaults(
query_engine_tools=[
list_tool,
vector_tool,
],
service_context=service_context,
select_multi=False
)Yeah the service_context is put everywhere
ow it seems the pydantic usage was the problem
ok also, the from documents option enabled, the answers of the queries are very bad/it does not answer at all
So I just use the class like this for example:vector_index = VectorStoreIndex(
documents,
service_context=service_context,
qa_text=qa_text
)
I don't think that is a possible kwarg to use -- it will get ignored 🤔
i think it was in your doc
I only saw for the moment the summary_text that can be used for summary index and tree index I think
for vectorstore I can't find out where, but if you know it does not work
how to proceed to make this work, maybe it can help for the filtering issue
btw, here is an example of logs of openAI, it seems that auto prompt in english are sended, and can cause the azure filtering to be weird:
So now let's see how to configure custom prompts if you have a solution for thisn, according to the prompts showed here
I you are always available, I let you tell me, else we can see later @Logan M
@sunnyb hmm, looking at the source code, that param is present, not sure why it's complaining about being missing. Will take a deeper look
