Can you share the error here please
TypeError: Parameters to generic types must be types. Got [<llama_index.indices.vector_store.base.VectorStoreIndex object at 0x7fab980425b0>, <llama_index.ind.
index = GPTListIndex[indices]
This is not correct, and likely where the error is
You should either create all those indexes as a single index to start with, or you can combine them with a router query engine or sub question query engine, and then use that query engine to create your chat engine
For example
query_engine = RouterQueryEngine.from_defaults(query_engine_tools)
chat_engine = ContextChatEngine.from_defaults(query_engine, llm=llm)
Contextchatengine is not in official doc
Se my problem is:
I have a number of collection in chromadb, I want to create one chat engine to connect all the collections
is using chromadb in this case advisable or should I use other collection
If we go through chroma tutorial, It picks one collection at a time and create indexes based on those.
how we combine multiple index to create one chatengine
from llama_index.memory import ChatMemoryBuffer
memory = ChatMemoryBuffer.from_defaults(token_limit=1500)
chat_engine = index.as_chat_engine(
chat_mode="context",
memory=memory,
system_prompt="You are a chatbot, able to have normal interactions, as well as talk about an essay discussing Paul Grahams life.",
)
you were talking about it?
Is the data different that you had to create different indexes?
yes different data connectors
Having multiple indexes calls for something like composable graph but I'm not sure whether it works with Chat engine or not.
Tried the same query on mendable and it gave me this
from llama_index.indices.composability import ComposableGraph
from llama_index.query_engine import ChatEngine
# Create the composable graph with your desired indices
graph = ComposableGraph.from_indices(
# Specify the indices and their summaries
SummaryIndex,
[index1, index2, index3],
index_summaries=[index1_summary, index2_summary, index3_summary],
storage_context=storage_context,
)
# Configure the chat engine to use the composable graph
chat_engine = ChatEngine(graph)
# Start a conversation with the chat engine
response = chat_engine.query("Can you tell me about XYZ?")
# Print the response
print(response)Do give it a try once, Else we gotta flash the light 🦇 in the sky to call @Logan M 🙂
from llama_index.query_engine import ChatEngine
This does not exists,.ImportError: cannot import name 'ChatEngine' from 'llama_index.query_engine' (/home/siddharth/.local/share/virtualenvs/ai-bot-1WLm1yfn/lib/python3.8/site-packages/llama_index/query_engine/init.py)
@WhiteFang_Jr
I was wondering if I can add all the vectors in the same index for different data connectors. What vector store will be good for this, or this is a bad idea?
Combining all the indexes can be tricky as you will have to check whether the vectors were made using same embedding model or not else there would be issue during query.
Can you not create the vectors at one place? OR
Create different chat engine for each index and answer that way?
Just checked, Composable graph only has as_query_engine method and nothing for Chat engine.
How do I enable chat continuation in subquestion query engine. By subquestion query engine I can connect all the indices ?
Else I am planning if I can create a simple chat engine to create chatengine like response and do chat continuation from the response I get from sunquestion query engine
your thoughts @WhiteFang_Jr @Logan M
You can create a sub-question query engine and give it to an agent for example
from llama_index.tools import QueryEngineTool, ToolMetadata
query_engine_tools = [
QueryEngineTool(
query_engine=sub_question_engine,
metadata=ToolMetadata(
name="query_engine",
description="Useful for asking questions about [your topic(s) here]",
),
),
]
agent = OpenAIAgent.from_tools(
query_engine_tools,
verbose=True,
# function_call="query_engine"
)By including
function_call it will force the agent to call the tool at least once on every user message -- you may or may not want this behaviour
I tried this insted of querying using the tools(query engine tools) it is searching for chat history and gives error as below:
pydantic.error_wrappers.ValidationError: 1 validation error for ChatMessage
content
str type expected (type=type_error.str)
@Logan M
you'll need to include more details here -- did you create a ChatMessage object somewhere?
Here's my code :
def get_indices_for_chat(user_id):
memory = ChatMemoryBuffer.from_defaults(token_limit=2500)
timea = time.time()
collections = UserIndexes.objects.filter(
user_id=user_id)
timeb = time.time()
print("filtering collections", timeb-timea)
indices = []
summary = []
for collection in collections:
if collection.file_name != "":
timec = time.time()
# index = get_indexes(collection).as_query_engine()
index = get_indexes(collection).as_query_engine()
# summary.append(collection.description)
query_engine_tool = QueryEngineTool(
query_engine=index,
metadata=ToolMetadata(
name=collection.file_name,
description=collection.description
),
)
indices.append(query_engine_tool)
timed = time.time()
print("getting index", timed-timec)
timee = time.time()
print("getting all indices", timee-timeb)
# graph = ComposableGraph.from_indices(
# SimpleKeywordTableIndex,
# indices,
# index_summaries=summary,
# max_keywords_per_chunk=50,
# )
# query_engine = SubQuestionQueryEngine.from_defaults(
# query_engine_tools=indices,
# service_context=get_service_context(),
# use_async=True,
# )
return indices
The above one sends the list of query engine tools and below code is used for openai agent:
@Logan M
index = get_indices_for_chat(user_id=user_id)
self.index = index
self.agent = OpenAIAgent.from_tools(
tools=self.index)
print("here")
# res = str(self.index.query(
# str(self.chat_engine.chat(message=get_prompt(message)))))
# print(res)
# final_res = str(self.chat_engine.chat(
# message=final_prompt(res, message), chat_history=self.messages))
res = self.agent.chat(message=message)
print(res)
res = self.agent.chat(message=message)
For this, you don't really need a kwarg, and message shouldjust be a string
res = self.agent.chat("Hello!")
If you are manually including the chat history, it needs to be a list of chat message objects
messages = [ChatMessage(role="assistant", content="Hello!")]
res = self.agent.chat("Hello!", chat_history=messages)
self.messages = [cm(content="Hello", role=MessageRole.ASSISTANT),]
res = self.agent.chat(message, chat_history=self.messages)
print(res)
thread = Thread.objects.get(id=thread_id)
self.messages += [
cm(content=message, role=MessageRole.USER),
# cm(content=res, role=MessageRole.ASSISTANT),
cm(content=res, role=MessageRole.SYSTEM),
]
still got same error
pydantic.error_wrappers.ValidationError: 1 validation error for ChatMessage
content
str type expected (type=type_error.str)
This means one of the content fields is not a string
So, something is wrong somewhere for that
but It did not query from the query engine provided in query engine tools all the indexes were in query engine tools
prompt: {"message":"extract unit price and unit cost of each item"}
res: To extract the unit price and unit cost of each item from the invoice data, I would need the specific format or structure of the invoice data. Could you please provide the invoice data or describe its structure? res
you'll need to provide better descriptions of the query engine tools. It picks a tool based on their descriptions
Thanks @Logan M , I will test with that.
It is not giving answer from the data it is trained with @Logan M
During indexing shall I keep all the data at one collection
Agents work by having a list of tools
On each user message, they can decide to use a tool based on the user message + tool descriptions, or they might choose to respond naturally. Thats just how it works.
Yea I think this was my initial suggestion 😆 embedding is cheap, easier to put everything in one index if that's what you want
I am using chromadb to store vectors, I tried this But it does not add the data , here's my code:
document = SimpleDirectoryReader(filepath).load_data()
docstore = MongoDocumentStore.from_uri(
uri=connection_string, namespace=user_id)
# create parser and parse document into nodes
# parser = SimpleNodeParser(
# chunk_size=1024, chunk_overlap=20
# )
node_parser = SimpleNodeParser.from_defaults(
chunk_size=1024, chunk_overlap=20)
nodes = node_parser.get_nodes_from_documents(document)
docstore.add_documents(nodes)
user_info = UserIndexes.objects.filter(user_id=user_id)[0]
collection = chroma_client.get_collection(user_info.collection)
vector_store = ChromaVectorStore(
chroma_collection=collection)
index_store = index_stores
# storage context initialization
storage_context = StorageContext.from_defaults(
vector_store=vector_store, index_store=index_store)
index = load_index_from_storage(service_context=get_service_context(
), storage_context=storage_context, index_id=user_info.index_id)
# print(len(document))
# print(document)
index.insert(document[0])
Suppose I add one document which is a essay of some topic, and 2nd I add another essay in the 3rd time it does not add the 3rd essay.
it does not give any error but while querying it says no information provided
Can i set the similarity k value to a higher number like 500-1000
You can, but won't that send a ton of text to the LLM for synthesis? Latency will be huge
index.as_query_engine(similarity_top_k=2) is the default
HI @Logan M
In the short term I have done in single index, in future is there any plan llma index will have function for suuquestion chat engine
Can we store data in gbs in one index
Yes, but also, not entirely advisable to do that? Once you have that much data, a) you'll probably want to use an optimized backend like qdrant, weaviate, or chroma. b) you might find that retrieval accuray becomes worse (lots of data, harder to find correct chunks). For that, you'll need to increase top k, or use hybrid search techniques (if the vector db offers that), or other retrieval tricks
I am currently using chroma
Then i will do some research on retrival tricks
