What is the advantage of the OpenAIAgent

SSaltuk

What is the advantage of the OpenAIAgent instead of a ContextChatEngine (besides function calling?). If i just want to perform plain RAG, so only ask questions against a set of data, does it make sense to use the OpenAIAgent?
I added langfuse for tracing and what i can see is that running the OpenAIAgent with a single tool for querying against the knowledge base has around 2x the latency of the pure ContextChatEngine.

13 comments

LLogan M

context chat engine just runs retrieval on every incoming user message, and stuffs that into the LLM input with the chat history

agents are more flexible, can break down questions into several sub-questions, re-word user messages, etc.

SSaltuk

How can it be then, every other setting is the same, my performance when evaluating with DeepEval is then worse when using the OpenAI Agent vs the ContextChatEngine?

SSaltuk

Plain Text

======================================================================

Overall Metric Pass Rates

Correctness (GEval): 40.00% pass rate
Answer Relevancy: 96.00% pass rate
Faithfulness: 68.00% pass rate
Contextual Precision: 44.00% pass rate
Contextual Relevancy: 32.00% pass rate
Contextual Recall: 84.00% pass rate

======================================================================

vs.

Plain Text

======================================================================

Overall Metric Pass Rates

Correctness (GEval): 68.00% pass rate
Answer Relevancy: 100.00% pass rate
Faithfulness: 88.00% pass rate
Contextual Precision: 48.00% pass rate
Contextual Relevancy: 40.00% pass rate
Contextual Recall: 96.00% pass rate

======================================================================

SSaltuk

First one is OpenAIAgent, second one is ContextChatEngine

SSaltuk

Every other tunable hyperparameter is the same, dataset also the same

LLogan M

Just a different approach. Like, it really matters what your questions are, etc.

LLogan M

I'm guessing your questions are relatively simple, so a context chat engine would do better

SSaltuk

Why exactly would a context chat engine be better in this case? Like what is the ratio for it?

LLogan M

Straightforward questions that have easy pieces of data to retrieve is my guess?

For example, try asking a followup question like "tell me more" and it probably won't work (or will hallucinate). That's actually why there is a CondensePlusQuestion chat engine

Anyways, use what works best you 😁 every option has its own pros and cons

SSaltuk

Why does a follow-up question not work with ContextChatEngine? ^^

LLogan M

Because the context engine is just retrieving based on the latest message. If it retrieves using the text "tell me more" it will likely retrieve text that's not at all helpful to the conversation

This is why while it is a bit slower, being able to reword the query is pretty powerful

SSaltuk

Okay then maybe can you help me with the following problem:
When i use the OpenAIAgent, the quality of the answers somehow gets worse, so it definitely feels like it answers the questions worse then the context chat engine. I can also try the CondensePlusQuestion chat engine, but i would have expected at least the same performance from the OpenAIAgent.

SSaltuk

What would i need to do to get the OpenAIAgents performance on par with the ContextChatEngine?

Add a reply

Find answers from the community

What is the advantage of the OpenAIAgent