I use `index.as_chat

At a glance

The community member is using index.as_chat_engine and memory, and they are trying to get the sources that have been used. However, they find that streaming_response.sources is empty. The comments suggest that streaming_response.source_nodes may contain the relevant information, but it is also empty.

The community members discuss different approaches, such as using index.as_query_engine.query(), which can retrieve the sources but does not relate to the chat history. They also provide an example of using OpenAIAgent and QueryEngineTool, which seems to work for them. The discussion also covers the difference between sources and source_nodes, with one community member explaining that sources refers to the tools used, while source_nodes refers to the nodes from the query engine tools.

There is no explicitly marked answer in the provided information.

TTsovak

I use index.as_chat_engine and memory. how to get sources that have been used?

Plain Text

streaming_response = as_chat_engine_var.stream_chat()

streaming_response.sources is en empty

13 comments

TTeemu

streaming_response.source_nodes

TTsovak

Attachment

TTsovak

it is empty

TTsovak

but it retrived a correct data and answer

TTsovak

if we use index.as_query_engine.query() then we can get sources. but this way doesn't relates to history. it is only for one question

LLogan M

If its empty, I think that means it didn't actually call a tool?

What chat engine are you using? Default?

It works for me

Plain Text

from llama_index.core import Document, VectorStoreIndex
from llama_index.core.tools import QueryEngineTool
from llama_index.agent.openai import OpenAIAgent

index = VectorStoreIndex.from_documents([Document.example()])

query_engine = index.as_query_engine()

tool = QueryEngineTool.from_defaults(query_engine, name='search', description='Useful for asking questions  about LLMs.')

agent = OpenAIAgent.from_tools([tool])


async def run():
  response = await agent.astream_chat("What are some facts about LLMs?")
  async for token in response.async_response_gen():
    print(str(token), end="", flush=True)
  print(response.source_nodes)
  print(response.sources)

if __name__ == "__main__":
  import asyncio
  asyncio.run(run())

LLogan M

sources is any tools, source nodes is nodes from query engine tools

TTsovak

the full code

Plain Text

 # Create a memory to store the chat history
        memory = ChatMemoryBuffer.from_defaults(llm=Settings.llm, token_limit=4095, chat_store=SimpleChatStore())

        # The simplest case
        as_chat_engine = self.index.as_chat_engine(
            # Select the best chat engine based on the current LLM.
            # Corresponds to `OpenAIAgent` if using an OpenAI model that supports
            # function calling API, otherwise, corresponds to `ReActAgent`
            chat_mode=ChatMode.BEST,
            memory=memory,
            streaming=True,
            vector_store_query_mode=VectorStoreQueryMode.SEMANTIC_HYBRID,
            similarity_top_k=3,
            text_qa_template=text_qa_template,
            refine_template=refine_template,
        )

TTsovak

inside of my code it actually calls OpenAIAgent.from_tools([tool])

TTsovak

your example is working well. thanks.

LLogan M

Sounds good 👍 if you also set verbose=True you can see when the agent makes a tool call

If there are no sources, it very likely did not make a tool call

TTsovak

it is very strange.
it calls

Plain Text

 return AgentRunner.from_llm(
                tools=[query_engine_tool],
                llm=llm,
                **kwargs,
            )

and inside this it calls:

Plain Text

 return OpenAIAgent.from_tools(
                tools=tools,
                llm=llm,
                **kwargs,
            )

in your example, it does almost the same. I cannot understand which arguments of function affects

LLogan M

Its the same thing, AgentRunner.from_llm() is just a helper that picks between openai agent and react agent

Add a reply

Find answers from the community

I use `index.as_chat_engine` and