hi, is there a way to make a query pipeline conditional? suppose i have a node A, if x=1 it should go to node B else it should go to node C.

Hi, I would like to wrap all this as a module in a query pipeline, how can i do it?

from llama_index.core.agent.react_multimodal.step import MultimodalReActAgentWorker
from llama_index.core.agent import AgentRunner
from llama_index.core.multi_modal_llms import MultiModalLLM
from llama_index.multi_modal_llms.openai import OpenAIMultiModal
from llama_index.core.agent import Task
from llama_index.core.schema import ImageDocument

mm_llm = OpenAIMultiModal(model="gpt-4-vision-preview", max_new_tokens=1000)

react_step_engine = MultimodalReActAgentWorker.from_tools(
[], # you can put some tools here
multi_modal_llm=mm_llm,
verbose=True,
)
agent = AgentRunner(react_step_engine)

query_str = "Your query string here"

image_document = ImageDocument(image_path="path_to_your_image.png")

task = agent.create_task(
query_str,
extra_state={"image_docs": [image_document]},
)

def execute_step(agent: AgentRunner, task: Task):
step_output = agent.run_step(task.task_id)
if step_output.is_last:
response = agent.finalize_response(task.task_id)
print(f"> Agent finished: {str(response)}")
return response
else:
return None

def execute_steps(agent: AgentRunner, task: Task):
response = execute_step(agent, task)
while response is None:
response = execute_step(agent, task)
return response

response = execute_steps(agent, task)

hi , i am following this guide about multi-modal reAct Agent:
https://docs.llamaindex.ai/en/latest/examples/multi_modal/mm_agent.html

but i get this error (image)

This is probably a stupid question but i dont know how to solve it. I am using llamaParse and then i am trying to upload the nodes into pinecone DB, but I am getting too much metadata with llamaParse so i am getting this error:

vector_store.add(nodes)

"Metadata size is 41191 bytes, which exceeds the limit of 40960 bytes per vector"

what can i do?

is there a way to select the number of retrieved nodes in an index different depending on the score? so that in a question if 5 nodes are very relevant include all them but in other question only one is relevant.

cohere_rerank = CohereRerank(api_key=api_key, model="rerank-multilingual-v2.0", top_n=2)

# Crear motor de chat con la colección seleccionada
chat_engine = index.as_chat_engine(
service_context=service_context,
similarity_top_k=10,
node_postprocessors=[cohere_rerank],
chat_mode="context",
memory=memory,
)

Hi, i am not able to use arize phoenix in llama 0.10 , what am i missing? https://colab.research.google.com/drive/1WGzquU46EGOg6goryL7UCwekWM7hUBz7?usp=sharing

Hey i am struggling with so many imports since llamaindex 0.10 https://colab.research.google.com/drive/1WGzquU46EGOg6goryL7UCwekWM7hUBz7?authuser=1#scrollTo=Edp6FrssvRI6

Hello, does someone know how to do this and what is the most optimal way to do it? thanks.

"how would you do an automatic summarize of the memory when it exceed the token_limit defined?"

Hello, is it possible to use define something like similarity_top_k=3, for the Chat Engine? I want to use the ChatMode but i am not sure how to control how many nodes to retrieve in each call

what is the estimate of when arize-phoenix will be available for llama 0.10?

Hello, I am using Arize-Phoenix for trazing, but i would like to know which user made each query. How can i do that? Thanks

Hello, is there a tutorial to do RAG over code in llamaindex? similar to this one in langchain: https://js.langchain.com/docs/use_cases/rag/code_understanding

Hello, is it possible to use define something like similarity_top_k=3, for the Chat Engine? I want to use the ChatMode but i am not sure how to control how many nodes to retrieve in each call

Hi! Maybe this question is stupid... but imagine i have 200 docs and i want to make a RAG to retrieve the documents where an specific topic is talked about. Imagine i have 200 documents abouts cars, but in only 20 of those they talk about ferraris.
I would like to do something like document comparison but at large scale. how can i do this? I would like to do something like analytics of documents with rag. I though about this (https://docs.llamaindex.ai/en/stable/examples/agent/agent_runner/agent_around_query_pipeline_with_HyDE_for_PDFs/ and https://docs.llamaindex.ai/en/stable/examples/agent/multi_document_agents/ )

but I would like to know if someone has already worked on this topic. Thanks.

@Logan M this cookbook does not work for the multimodal for haiku: https://github.com/run-llama/llama_index/blob/main/docs/cookbooks/anthropic_haiku.ipynb

"ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
llama-index-llms-anthropic 0.1.6 requires anthropic<0.21.0,>=0.20.0, but you have anthropic 0.17.0 which is incompatible."

Hi guys, based on your experience which reranking model is better ? Colbert or Cohere?

is there an example of how to include an agent as a part of a query pipeline?

so, imagine i have like 10-20 modules, and i want to make a retry loop with an agent but just on an specific part, not the whole pipeline.

qp = QP(
modules={
"input": InputComponent(),
...
"agent_input_component": agent_input_component,
...
"agent_output_component": agent_output_component,
},

@Logan M is it possible to use this pipeline but with a chat.engine? I want to use this type of pipeline but i also want to have a memory

retriever = index.as_retriever(similarity_top_k=6)
summarizer = TreeSummarize(
service_context=ServiceContext.from_defaults(
llm=OpenAI(model="gpt-3.5-turbo")
)
)
reranker = CohereRerank()

p = QueryPipeline(verbose=True)
p.add_modules(
{
"input": InputComponent(),
"retriever": retriever,
"summarizer": summarizer,
}
)
p.add_link("input", "retriever")
p.add_link("input", "summarizer", dest_key="query_str")
p.add_link("retriever", "summarizer", dest_key="nodes")

output = p.run(input="MY QUESTION")

print(str(output))

how to do reranking with chatengine?

Hello,

is there a function similar to langchain's ConversationSummaryBufferMemory? @Logan M

Hello, is there a way to load into the memory a summarized text of its contents? I tried this code:

from llama_index.llms.base import ChatMessage

Load the chat history from memory

chat_history = memory.get_all()
print("chat_history",chat_history)

Concatenate text from chat history

text_of_memory = " ".join([message.content for message in chat_history])
print("text_of_memory",text_of_memory)

Summarize the chat history

summary_response = await summarizer.aget_response("Summarize chat", [text_of_memory])
print("summary_response",summary_response)

Clear the memory

chat_engine.reset()

Create a ChatMessage with the summary

summary_message = ChatMessage(content=summary_response, role="system")
print("summary_message",summary_message)

Insert the summary message into the memory

memory.insert(summary_message)
print("memory",memory)

It works until the memory.insert:

chat_history [ChatMessage(role=<MessageRole.USER: 'user'>, content='hello', additional_kwargs={}), ChatMessage(role=<MessageRole.USER: 'user'>, content='hello', additional_kwargs={}), ChatMessage(role=<MessageRole.USER: 'user'>, content='hello', additional_kwargs={})]
text_of_memory hello hello hello
1 text chunks after repacking
summary_response The chat appears to consist of the repeated the word "hello" multiple times.
summary_message system: The chat appears to consist of the repeated phrase "como hacer la peticion" (how to make the request) multiple times.
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-15-6d2d42d50f09> in <cell line: 19>()
17 print("summary_message",summary_message)
18 # Insert the summary message into the memory
---> 19 memory.insert(summary_message)
20 print("memory",memory)

AttributeError: 'ChatMemoryBuffer' object has no attribute 'insert'