LlamaIndex ChatEngine

I'm trying to use Llamaindex without a full RAG scenario. I.e. query the LLM as a first try, but then do an internet search when it's clear that the information isn't part of it's trained data (e.g. the weather in London today). I was thinking of using both as separate query engines and then chaining them together. However, I can't yet get a simple direct chat (via LlamaIndex) to work.

The problem is that I don't need a RAG scenario. I can't see from the documentation how I might do this with LlamaIndex (I'm using a local LLM, but this should not be an issue).

Langchain in theory is better at pulling together different things without assuming a RAG use-case. Unfortunately, Langchain, is very buggy, the documentation is terrible, and out of date, and the code-base keeps changing faster than anyone can keep up with the bugs.

In short, I can think of only one thing that I like about langchain, and that is that it doesn't assume RAG.

Is it possible to do this with LlamaIndex as well? This isn't obvious (to me) in the documentation.

I've looked through previous questions around metadata searching, but can't seem to find a decent source to help me get what I need. I'm currently splitting my searching through a direct chromadb search, and a llamaindex search. If the user only want's the actual chunk source, then I use chromadb directly. If they want an interpretation through an LLM narrator, then I pass it to llamaindex.

I can constrain which specific source files are searched with chromadb using the following

print(f"****************** Only searching throgh documents {resourceArray}") results = collection.query( query_texts=[queryText], where = { "file_name": { "$in": resourceArray } }, include=["metadatas", "documents", "distances"], n_results=3, )

In this case, resourceArray simply holds the original file names for the content that was ingested.

I can't work out how to do this with llamaindex. I've see this source example, but can't work out how to use a list filter along the same lines as chromadb.

query_engine = index.as_query_engine( service_context=service_context, similarity_top_k=3, vector_store_query_mode="default", filters=MetadataFilters( filters=[ ExactMatchFilter(key="name", value="paul graham"), ] ), alpha=None, doc_ids=None, )

Have I missed something?

Hi - I'm trying to get some query_engine sample code to work from this page: https://docs.llamaindex.ai/en/stable/examples/usecases/10k_sub_question.html

llama-index is version 0.9.13

One key difference that I have with the sample and my code is that I am using a local LLM, and not OpenAI. However, I am setting a local dummy key and an alternative base_url, which points to my local server. My embedding model is also a local HF one. It seems to build the indexes fine, and make a call to the local LLM, but the query engine seems to fail.

The error is "ValueError: Expected tool_calls in ai_message.additional_kwargs, but none found.".

Source and error trace in the included error.txt file

Uber/Lyft EBITDA/Financials etc. are completely outside the scope of the domain I'm querying.

And these examples are reasonably confusing the question process. So I see prompts being sent to the LLM about financials for MSFT/GOOG. This makes sense if those examples are sent, but counterproductive.

Is there any way to avoid this example stage, or provide an alternative set of examples?

Potential Bug?

I've been working on something that should compare content across multiple documents (i.e an index per document set). When I trace the queries that are being sent to the local LLM, there are some some example questions also passed along with the prompt:

It' now clear why tree_summarize goes back to the LLM so many times

I'm trying to work out how to add a Google Search into a query chain.

I can see some example code here (https://github.com/run-llama/llama-hub/blob/main/llama_hub/tools/notebooks/google_search.ipynb) that shows calling Google to provide some initial information that is then fed to the LLM. However, I want to include this as part of a chain (e.g. a query engine).

Is there any example of how I could use Google as part of a RAG query where it is just another index source?