Find answers from the community

Updated 3 months ago

How to logger when calling a query through a agent

How to logger when calling a query through a agent
1
k
c
L
34 comments
Please stand by @cK5150 as I search the knowledge base. We are experiencing especially high traffic right now and might take longer than usual. :wave:
----------------------
To set up logging when calling a query through an agent, you can follow the example from the Langchain Demo notebook (https://github.com/jerryjliu/gpt_index/tree/main/examples/langchain_demo/LangchainDemo.ipynb). First, you need to set up logging:

Plain Text
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))


Then, you can initialize the agent with the GPT Index as a memory module or a callable tool. Here's an example of using GPT Index as a memory module:

Plain Text
from langchain import OpenAI
from langchain.agents import initialize_agent
from gpt_index import GPTListIndex, GPTIndexMemory

index = GPTListIndex([])
memory = GPTIndexMemory(index=index, memory_key="chat_history", query_kwargs={"response_mode": "compact"})
llm = OpenAI(temperature=0)
agent_chain = initialize_agent([], llm, agent="conversational-react-description", memory=memory)


Now, when you call a query through the agent, the logging will be enabled:

Plain Text
agent_chain.run(input="what's my name?")


Remember that this example assumes you have already set up the GPT Index and Langchain. Make sure to follow the necessary steps from the provided extracts to set up both components correctly.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
@kapa.ai give an example with a callable tool
Let me check the knowledge base for you. One moment! :books:
----------------------
To set up logging when calling a query through an agent using GPT Index as a callable tool, you can follow the example from the Langchain Demo notebook (https://github.com/jerryjliu/gpt_index/tree/main/examples/langchain_demo/LangchainDemo.ipynb). First, set up logging:

Plain Text
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))


Next, initialize the agent with the GPT Index as a callable tool:

Plain Text
from langchain.agents import Tool
from langchain.chains.conversation.memory import ConversationBufferMemory
from langchain import OpenAI
from langchain.agents import initialize_agent
from gpt_index import GPTSimpleVectorIndex

index = GPTSimpleVectorIndex.load_from_disk('../vector_indices/index_simple.json')

tools = [
    Tool(
        name="GPT Index",
        func=lambda q: str(index.query(q)),
        description="useful for when you want to answer questions about the author. The input to this tool should be a complete English sentence.",
        return_direct=True
    ),
]

memory = ConversationBufferMemory(memory_key="chat_history")
llm = OpenAI(temperature=0)
agent_chain = initialize_agent(tools, llm, agent="conversational-react-description", memory=memory)


Now, when you call a query through the agent using the callable tool, the logging will be enabled:

Plain Text
agent_chain.run(input="What did the author do growing up?")


Remember that this example assumes you have already set up the GPT Index and Langchain. Make sure to follow the necessary steps from the provided extracts to set up both components correctly.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
@kapa.ai With logging enabled how do I see all inputs/outputs
Let me check the knowledge base for you. One moment! :books:
----------------------
To see all inputs and outputs with logging enabled, you can create a custom logging function that logs the input and output of the agent. Here's an example based on the Langchain Demo notebook (https://github.com/jerryjliu/gpt_index/tree/main/examples/langchain_demo/LangchainDemo.ipynb) using GPT Index as a callable tool:

First, set up logging:

Plain Text
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))


Next, initialize the agent with the GPT Index as a callable tool:

Plain Text
from langchain.agents import Tool
from langchain.chains.conversation.memory import ConversationBufferMemory
from langchain import OpenAI
from langchain.agents import initialize_agent
from gpt_index import GPTSimpleVectorIndex

index = GPTSimpleVectorIndex.load_from_disk('../vector_indices/index_simple.json')

tools = [
    Tool(
        name="GPT Index",
        func=lambda q: str(index.query(q)),
        description="useful for when you want to answer questions about the author. The input to this tool should be a complete English sentence.",
        return_direct=True
    ),
]

memory = ConversationBufferMemory(memory_key="chat_history")
llm = OpenAI(temperature=0)
agent_chain = initialize_agent(tools, llm, agent="conversational-react-description", memory=memory)


Create a custom function to log input and output:

Plain Text
def log_input_output(agent, input_text):
    logging.info(f"Input: {input_text}")
    output = agent.run(input=input_text)
    logging.info(f"Output: {output}")
    return output


Now, when you call a query through the agent using the custom function, the logging will show the input and output:

Plain Text
log_input_output(agent_chain, "What did the author do growing up?")
This example assumes you have already set up the GPT Index and Langchain. Make sure to follow the necessary steps from the provided extracts to set up both components correctly.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
@kapa.ai how to use llama logger when running a index.query through a agent
Let me check the knowledge base for you. One moment! :books:
----------------------
To use Llama Logger when running an index.query through an agent, you can follow these steps:

First, set up logging:

Plain Text
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))


Next, initialize the agent with the GPT Index as a callable tool:

Plain Text
from langchain.agents import Tool
from langchain.chains.conversation.memory import ConversationBufferMemory
from langchain import OpenAI
from langchain.agents import initialize_agent
from gpt_index import GPTSimpleVectorIndex

index = GPTSimpleVectorIndex.load_from_disk('../vector_indices/index_simple.json')

tools = [
    Tool(
        name="GPT Index",
        func=lambda q: str(index.query(q)),
        description="useful for when you want to answer questions about the author. The input to this tool should be a complete English sentence.",
        return_direct=True
    ),
]

memory = ConversationBufferMemory(memory_key="chat_history")
llm = OpenAI(temperature=0)
agent_chain = initialize_agent(tools, llm, agent="conversational-react-description", memory=memory)


Now, when you call a query through the agent using the callable tool, the Llama Logger will log the information:

Plain Text
agent_chain.run(input="What did the author do growing up?")


This example assumes you have already set up the GPT Index and Langchain. Make sure to follow the necessary steps from the provided extracts to set up both components correctly.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------
As long as llama logger is passed into the service context of the index, it should pick everything up.

Just make sure if load from disk that you still pass in the service context
llama_logger = LlamaLogger()
service_context = ServiceContext.from_defaults(llama_logger=llama_logger)
index = GPTSimpleVectorIndex.load_from_disk("index.json",service_context=service_context)
Yup! Then you can call llama_logger.get_logs() to get the logs after a query
I keep getting this error. ValueError: Cannot use llm_token_counter on an instance without a service context.
maybe also pass the service_context into the query? index.query(..., service_context=service_context)

not sure what's going on there. If that doesn't work, maybe share the full stack trace/code lol

Load the GPTSimpleVectorIndex only once

llama_logger = LlamaLogger()
service_context = ServiceContext.from_defaults(llama_logger=llama_logger)
index = GPTSimpleVectorIndex.load_from_disk("index.json",service_context=service_context)

tools = [
Tool(
name="GPT Index",
func=lambda q: str(index.query(q)),
description="useful for when you want to answer questions about the author. The input to this tool should be a complete english sentence.",
return_direct=True,
),
]
conv_window_memory = ConversationBufferWindowMemory(k=3, memory_key="chat_history")
llm = ChatOpenAI(temperature=0.5, model_name="gpt-3.5-turbo")
agent_chain = initialize_agent(
tools, llm=llm, agent="conversational-react-description", memory=conv_window_memory, verbose=True
)

app = Flask(name, template_folder="templates")
CORS(app)

@app.route("/")
def home():
return render_template("index.html")

@app.route("/api/ask-ai", methods=["POST"])
def api_ask_ai():
query = request.json.get("query", "")

response = agent_chain.run(
input=f"{{GPT Index: {query}}}",
)
# Update the memory with the actual response
memory_variables = conv_window_memory.load_memory_variables({})
pprint(memory_variables)
return jsonify({"response": response})

if name == "main":
app.run()
Stack trace too long for discord. Thank you so much for your help.
Entering new AgentExecutor chain...
Thought: Do I need to use a tool? Yes
Action: GPT Index
Action Input: What is contentment?ERROR:app_v2:Exception on /api/ask-ai [POST]
Traceback (most recent call last):
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\flask\app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\flask\app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\flask_cors\extension.py", line 165, in wrapped_function
return cors_after_request(app.make_response(f(*args, kwargs))) File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\flask\app.py", line 1823, in full_dispatch_request rv = self.dispatch_request() File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\flask\app.py", line 1799, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(view_args)
File "C:\Users\peace\Documents\OpenAI\Zeteo\app_v2.py", line 77, in api_ask_ai
response = agent_chain.run(
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\chains\base.py", line 216, in run
return self(kwargs)[self.output_keys[0]]
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\chains\base.py", line 116, in call
raise e
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\chains\base.py", line 113, in call
outputs = self._call(inputs)
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\agents\agent.py", line 792, in _call
next_step_output = self._take_next_step(
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\agents\agent.py", line 695, in _take_next_step
observation = tool.run(
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\tools\base.py", line 73, in run
raise e
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\tools\base.py", line 70, in run
observation = self._run(tool_input)
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\langchain\agents\tools.py", line 17, in _run
return self.func(tool_input)
File "C:\Users\peace\Documents\OpenAI\Zeteo\app_v2.py", line 37, in <lambda>
func=lambda q: str(index.query(q)),
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\llama_index\indices\base.py", line 255, in query
return query_runner.query(query_str)
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\llama_index\indices\query\query_runner.py", line 349, in query
return query_combiner.run(query_bundle, level)
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\llama_index\indices\query\query_combiner\base.py", line 66, in run
return self._query_runner.query_transformed(
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\llama_index\indices\query\query_runner.py", line 209, in query_transformed
return query_obj.query(query_bundle)
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\llama_index\token_counter\token_counter.py", line 77, in wrapped_llm_predict
with wrapper_logic(_self):
File "C:\Users\peace\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 135, in enter
return next(self.gen)
File "C:\Users\peace\Documents\OpenAI\Zeteo\myenv\lib\site-packages\llama_index\token_counter\token_counter.py", line 42, in wrapper_logic
raise ValueError(
ValueError: Cannot use llm_token_counter on an instance without a service context.
Change this line

func=lambda q: str(index.query(q)),

to

func=lambda q: str(index.query(q, service_context=service_context)),

(Also, you might get better results if you change the agent_chain.run to just response = agent_chain.run(input=query) , but maybe not either haha)
Same error with both options.
I appreciate your help even if it doesn't work. Thank you.
My goal is to just see the exact prompt being sent to the LLM (openAI in this case) so I can reaffirm the bot is seeing exactly what I think it is seeing. Right now I just see
Thought: Do I need to use a tool? Yes
Action: GPT Index
Action Input: Who is the author?
And then the response
which llama_index version do you have? Maybe try upgrading? I just tried this and it worked

Plain Text
from llama_index import GPTSimpleVectorIndex, SimpleDirectoryReader, ServiceContext
from llama_index.logger import LlamaLogger

docs = SimpleDirectoryReader('./data').load_data()

service_context = ServiceContext.from_defaults(llama_logger=llama_logger)

index = GPTSimpleVectorIndex.from_documents(docs, service_context=service_context)

response = index.query("What did the author do growing up?", service_context=service_context)

llama_logger.get_logs()

You can also try enabling debug logs, but this doesn't always work for some reason

Plain Text
import logging
import sys

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
I'm using llama_index-0.5.16 I upgraded to 0.5.21 and my index broke. I will continue forward, thank you!
for me helped importing index from llama-index, NOT from gpt-index
Add a reply
Sign up and join the conversation on Discord