Find answers from the community

Home
Members
elmegatan26
e
elmegatan26
Offline, last seen 3 months ago
Joined September 25, 2024
I am using react chat engine and want to override MessageRole.System when submitting a query. E.g. default from llama_debug
Plain Text
ChatMessage(role=<MessageRole.SYSTEM: 'system'>, content='\nYou are designed to help with a variety of tasks, from answering questions...
5 comments
e
L
From the help docs https://gpt-index.readthedocs.io/en/v0.7.16/examples/chat_engine/chat_engine_context.html can someone point me to an example of updating memory = ChatMemoryBuffer.from_defaults(token_limit=1500) with a custom interaction?
11 comments
L
e
Upgrading from .6.38 to .7.9. Getting this error when submitting question.
Plain Text
File "/app/./index_server.py", line 197, in query_index
    chat_engine = index.as_chat_engine(chat_mode='react',
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llama_index/indices/base.py", line 383, in as_chat_engine
    return ReActAgent.from_tools(
           ^^^^^^^^^^^^^^^^^^^^^^
TypeError: ReActAgent.from_tools() got an unexpected keyword argument 'similarity_top_k'
Any direction will be greatly appreciated. Some of the migration docs appear to be 404.
5 comments
L
k
e
I'm upgrading from .6 to .7 today and the token_counter migration guide is a 404. Anyone know where to find? https://gpt-index.readthedocs.io/how_to/callbacks/token_counting_migration.html
4 comments
e
L
From https://gpt-index.readthedocs.io/en/latest/examples/callbacks/TokenCountingHandler.html Is the tokenizer accurate if you use gpt-turbo3.5-turbo-16k like so?
Plain Text
token_counter = TokenCountingHandler(
    tokenizer=tiktoken.encoding_for_model("gpt-3.5-turbo-16k").encode
)

callback_manager = CallbackManager([token_counter])

llm_predictor = LLMPredictor(
    llm=ChatOpenAI(model_name='gpt-3.5-turbo-16k', temperature=0)
)
3 comments
e
L
Can anyone direct me to an example of how to use TokenCountingCallback?
11 comments
L
e
I am indexing files when starting an application using
Plain Text
documents = SimpleDirectoryReader(documents_dir, recursive='true', filename_as_id=True, num_files_limit=10).load_data()
This is working as expected. Docs are being split up and doc ids are being assigned to the source documents (e.g. my-doc.md_part_1, my-doc.md_part_2, etc). I'm having a problem when uploading new documents using
Plain Text
document = SimpleDirectoryReader(input_files=[doc_text]).load_data()[0]
in that case, only a small portion of the document is being indexed (e.g. my-doc.md). Am I missing a detail to properly chunk the single document and include all parts in the index?
7 comments
L
e
Testing out CondenseQuestionChatEngine. I'm getting this error when submitting a question:. Does anyone have an example of where you set the mode? I'm not finding it in the docs.
Plain Text
raise ValueError(f"Unknown chat mode: {chat_mode}")
ValueError: Unknown chat mode: <llama_index.chat_engine.condense_question.CondenseQuestionChatEngine object... >
8 comments
L
e
I'm upgrading an app that uses llama-index==0.5.40 to llama-index==0.6.11. Using the example below, the entire contents of a 3 page test PDF are added to the index in the 0.5.4 version, but only the first page in 0.6.11. I've looked through the docs but any hints or suggestions are welcome. With the upgrade I also updated PyPDF2==3.0.1. to pypdf==3.9.0
Plain Text
index = GPTVectorStoreIndex([], service_context=service_context)
document = SimpleDirectoryReader(input_files=[doc_text]).load_data()[0]
index.insert(document)
2 comments
e
L
I am using
Plain Text
chat_engine = index.as_chat_engine(
                    chat_mode="context",
                    system_prompt="You are a chatbot with access to extra context information about dogs. Only answer questions if you can find the answer in the context, otherwise, kindly inform the user that you cannot find the answer.",
                    chat_history=custom_chat_history
    )
All is working well and the LLM only answers questions about "dogs". My issue is that when I introduce the recent chat history, if someone asks a question about "cats", the LLM properly handles the quesiton the first try but any follow-up question including cats will be answered. Any recommendations for handling this situation?
11 comments
e
L
I am looking to upgrade from Llamaindex 0.7.6 to 0.7.13 and having a problem with the AI response from gpt-3.5-turbo-16k. I include a custom chat history interaction and a custom prompt that instructs the engine to only answer questions relevant to the index. When updating to 0.7.13 my custom instructions are largely ignored and the AI will answer questions not included in the index. This does not happen with 0.7.6.
Plain Text
chat_engine = index.as_chat_engine(
  chat_mode='react', 
  chat_history=custom_chat_history,
  similarity_top_k=max_top_k,
  refine_template=DEFAULT_REFINE_PROMPT, 
  text_qa_template=custom_prompt)

If anyone has some hints to hunt down why this happens it will be greatly appreciated.
5 comments
L
K
e
When querying OpenAI gpt-3.5-turbo I'm getting back responses that include "The provided context is not relevant to the original question" and "Therefore, the original answer remains the same:" followed by an answer. Can someone direct me on how to stop that? I'm using latest Lllamindex with
Plain Text
 llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo", max_tokens=2048))
    service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, chunk_size=1024)
5 comments
e
a
Does anyone know of a way to automatically pass in a filename as doc_id when using SimpleDirectoryReader? Kapa AI believes there is a way to use use_filename_as_id=True but it is not working E.g.
Plain Text
# Load documents with filenames as document IDs
documents = SimpleDirectoryReader('path/to/your/data', use_filename_as_id=True).load_data()
5 comments
L
e
Using 0.6.14. Does
Plain Text
GPTVectorStoreIndex.from_documents
support deleting docs from the index? I am passing in a filename as doc_id when inserting and passing it back in to
Plain Text
index.delete(<id>)
but the index does not remove the file. I've tried the actual doc_id from
Plain Text
source_nodes.node.doc_id
and
Plain Text
source_nodes.node.relationships.1
but nothing is working. Any troubleshooting tips would be appreciated.
6 comments
L
e