Find answers from the community

Home
Members
jma7889
j
jma7889
Offline, last seen 3 months ago
Joined September 25, 2024
j
jma7889
·

Responw

I am using retriever with top k = 10 and a generic responseSynthesizer, i can see the result is in node 5, however no matter what mode I use (I tried all of them), the result is always wrong (n/a mostly, and a couple of wrong answers). But, if I use a keyword post processor to narrow the results to 1 node, then the same responseSynthesizer returns the correct answer. I suspect if I ask LLM with each node, it will have one correct answer. Not sure why llama index synthesizer mostly gives N/A.
What should I do to improve the responseSynthesizer quality?
7 comments
j
L
Does anyone know how to fix the maximum recursion depth exceeded in comparison error in DocumentsSummaryIndex?
Plain Text
---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
Cell In[4], line 87
     84 _, _, scope_id = question
     85 if not '>' in scope_id or is_child_of_yes_scope(scope_id):
     86     #print(f"query scope_id={scope_id}, {question}")
---> 87     result: Result = query_engine_manager.execute_questions([question], None, None)           
     88     # Check the result and update yes_scopes if result starts with 'yes'
     89     if result.question_answer_pairs:

File ~/dev/airpunchai/annotation-assistant/app/llm/model_context.py:542, in QueryEngineManager.execute_questions(self, annotator_questions, taxonomy_map, flatten_org_taxonomy)
    539 logger.debug("query_template_size=%d", len(query_template))
    540 logger.debug("query_template=%s", query_template)
--> 542 query_result = self.query_engine.query(query_template)
    544 # Create a QuestionAnswer object for the current iteration
    545 qa = QuestionAnswer(
    546     question_id=question_id,
    547     taxonomy=taxonomy,
   (...)
    550     taxonomy_answer=[],  # Assuming this is an empty list for now, modify as needed
    551 )

File ~/Library/Caches/pypoetry/virtualenvs/chatgpt-retrieval-plugin-g8Qw76ZE-py3.10/lib/python3.10/site-packages/llama_index/indices/query/base.py:23, in BaseQueryEngine.query(self, str_or_query_bundle)
     21 if isinstance(str_or_query_bundle, str):
     22     str_or_query_bundle = QueryBundle(str_or_query_bundle)
...
    117 def __instancecheck__(cls, instance):
    118     """Override for isinstance(instance, cls)."""
--> 119     return _abc_instancecheck(cls, instance)

RecursionError: maximum recursion depth exceeded in comparison
18 comments
M
L
j
j
jma7889
·

Openai

Hi all. Does llama-index handle the OpenAI API retry in query engine for RateLimitError? I see some code in the repo openai_utils.py to handle the retry. But I also see llama-index uses lang chain openai api wrapper which seems throwing RateLimitError out. Should I handle it in my code?
4 comments
j
L
j
jma7889
·

Pypdf

Bug? I realized the pdf file no longer working after the llama index changed from pypdf2 to pypdf, https://github.com/jerryjliu/llama_index/blob/main/llama_index/readers/file/docs_reader.py?#L21 It throws error 'pkg pypdf not found', i have tried both poetry add and upgrade for pypdf. It still cannot be imported. Is this a bug? Why change from pypdf2 to pypdf?
14 comments
L
j
chroma quality is so bad, how to improve? Hi folks, I am working on a document q&a project. I was using GPTTreeIndex with child branch factor of 2 before, it was very good. Now I switched to chroma vector store. The result is so bad that chroma results look like from an idiot. Does anyone know how to improve it? e.g. change embedding, or feed the collection to GPTTreeIndex? I did try top k from 1 to 5, the results are more or less like from idiot number 1 to idiot number 5, with minor improvements only. For both indexes, I use the same model settings with gpt3.5
11 comments
L
a
j
Hello. I have a design question on using Azure AI Search for RAG. I have a large set of document in Azure AI Search that are stored with searchable key doc_id, text chunks, and their embedding. How to use llama index to: query "foo" for embedding only within scope of doc_id="XYZ"? The hybrid mode is not designed for this case. Should I create custom retriever? What is the right approach? Any existing examples? I guess this use case is common to all vector store based RAG. The key is to get the nodes with "XYZ" first then build retreiver with those nodes I guess.
3 comments
j
D
j
jma7889
·

Release

why poetry version shows 0.10.4 but the latest tag is 0.10.3?
2 comments
L
j
jma7889
·

0.10.x

from https://docs.llamaindex.ai/en/stable/getting_started/starter_example.html
Plain Text
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

cannot import, so changed to
Plain Text
from llama_index.core.indices import VectorStoreIndex
from llama_index.core.readers import SimpleDirectoryReader

content_file ="myfile"

documents = SimpleDirectoryReader(content_file).load_data()
index = VectorStoreIndex.from_documents(documents)

got runtime error
Plain Text
----> 1 from llama_index.core.indices import VectorStoreIndex
...
......./llama_index/core/indices/composability/graph.py:5
      1 """Composability graphs."""
      3 from typing import Any, Dict, List, Optional, Sequence, Type, cast
...
----> 6 from llama_index.core.llms import ChatMessage, MessageRole
      7 from llama_index.core.utils import get_tokenizer
     10 class TokenCounter:

ImportError: cannot import name 'ChatMessage' from 'llama_index.core.llms' (unknown location)
5 comments
j
L
Hi @Logan M , I haven't upgraded llama index for a month, good to see so many new features. Today, I tried to upgrade, I have an issue with 0.8.0 to 0.8.5.
For my Q&A feature that uses Tree GPT35turbo before, it worked well up to 0.7.24, starting 0.8.0, it returns a lot more false positive results. For example, instead of having 10 yes for a bunch of questions, I saw 50 yes now, 40 were false positive. Do I need to change anything for the breaking changes? I have been setting my own defaults with gpt3.5turbo, temperature of 0.01 and tree retriever, so the defaults change should not impact me. But the prompts changes do, to me, new tree prompts do not work as well as before. Please advice. Thanks in advance.
5 comments
j
L