LlamaIndex
Log in

Find answers from the community

Home
Members
andreas_martinson
a
andreas_martinson
Offline, last seen last week
Joined September 25, 2024
aandreas_martinson
·

Challenges with using Claude Sonnet 3.5 and ReactAgent

Anyone found that while using Claude Sonnet 3.5 it doesn't seem to work very well with the ReactAgent? It will hallucinate many steps at once so it says,

Plain Text
Action:
Thought:

Action:
Thought:
Observation:


Working through it, but wow. Just doesn't seem to work too well
14 comments
L
a
aandreas_martinson
·

Alright, so I am comparing

Alright, so I am comparing SQLAutoVectorQueryEngine to using the OpenAIAgent class. Even when I pass in the SQLAutoVectorQueryEngine as a tool to the OpenAI agent, the OpenAI agent doesn't perform as well. Any insights into the differences between the two? How can I narrow down why one performs better than the other?

From what I can see, the OpenAI agent is actually trying to answer the query itself and then passing the query into the tool. Which is not what I want 😮‍💨 I would rather the chatbot just allow the tool to create the query in the first place.

Also, if I just want a chatbot, but with the accuracy of the SQLAutoVectorQueryEngine, do I pass in an agent as one of its tools? That will help in the cases the user is chatting without asking for anything needed in the vector or sql databases
2 comments
a
aandreas_martinson
·

Multiple SQL Tables

I managed to figure out how to use retrieval with a database to avoid overflowing the context window, however, I don't know if this is the recommended method. Could someone let me know if this is the optimal approach if you're not passing in a set list of tables?

Plain Text
from llama_index.indices.struct_store.sql_query import (
    SQLTableRetrieverQueryEngine,
)
from llama_index.objects import (
    SQLTableNodeMapping,
    ObjectIndex,
    SQLTableSchema,
)
from llama_index import VectorStoreIndex

table_node_mapping = SQLTableNodeMapping(sql_database)

from llama_index.retrievers import NLSQLRetriever

# default retrieval (return_raw=True)
nl_sql_retriever = NLSQLRetriever(sql_database, return_raw=True)

# Get all tables wrapped as SQLTableSchema Objects
table_schema_objs = nl_sql_retriever._get_tables("")

obj_index = ObjectIndex.from_objects(
    table_schema_objs,
    table_node_mapping,
    VectorStoreIndex,
)

query_engine = SQLTableRetrieverQueryEngine(
    sql_database, obj_index.as_retriever(similarity_top_k=1)
)

response = query_engine.query("How many albums do the rolling stones have?")
print(response)


Specifically this line nl_sql_retriever._get_tables("") I'm using it to get the list of tables wrapped in the SQLTableSchema class, but it might exist somewhere else more conveniently.

I was referencing this tutorial: https://docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/SQLIndexDemo.html

Also, a side-question, I found this old tutorial that looks like its been deleted. Not sure if any part of this is relevant anymore either using the context builder: https://gpt-index.readthedocs.io/en/v0.6.22/examples/index_structs/struct_indices/SQLIndexDemo-ManyTables.html

Anyway, any help is appreciated! Thanks
1 comment
r