Hi guys, i'm running into an issue in production, i can't tell if this because my chunks are too big when trying to get embeddings or if my batch size is too big in qdrant client. (right now the batch size is 16). If the chunks are too big shouldn't that be handled by the NodeParser, I'm currently using SemanticSplitterNodeParser?

"2024-07-17 14:03:22,630 - ERROR - 13 - ThreadPoolExecutor-0_0 - root - index_asset - index_asset.py:39 - index_asset() >>> Error indexing asset into Qdrant: Error code: 400 - {'error': {'message': "This model's maximum context length is 8192 tokens, however you requested 10125 tokens (10125 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.", 'type': 'invalid_request_error', 'param': None, 'code': None}}"

i'm trying to run a custom metadata extractor i built in an ingestion pipeline. Using GPT4o works fine but when i try and use anthropic LLM i get 'Anthropic' object has no attribute '_aclient'. Is this just not implemented or am i doing something wrong?

I'm using vector store index with hybrid search using qdrant and the process is running on a google cloud run, rather than have the ONNX model load in memory and redownload every time it boots up i wanted to save the model to a volume drive i mounted to the cloud run instance. Is there a way to load and save the sparse embedding generator to the volume mount (and also have it load the model from there)?

HI team, I'm trying to retrieve the source nodes and node ids from the sub questions in the sub question query engine, i see that this notebook shows how to get the sub query questions but what about the source nodes for those questions?

https://github.com/jerryjliu/llama_index/blob/main/docs/examples/query_engine/sub_question_query_engine.ipynb

Anybody else getting Pydantic errors with the new release?

WARNING 2023-09-21 19:29:08 llama_index.response_synthesizers.refine [refine.py:refine:181] [_give_response_single] [P:97842] Validation error on structured response: 1 validation error for StructuredRefineResponse
root
StructuredRefineResponse expected dict not NoneType (type=type_error)

hi guys when using any refine prompt sometime the output ends up being "The new context does not provide any additional information that would require a refinement of the original answer. The original answer remains accurate and complete."
rather than providing the original response. Any idea whats happening here? its happening both on the Tree and The Simple vector
------------
Given the new context, refine the original answer to better answer the question. If the context isn't useful, output the original answer again.
DEBUG:llama_index.indices.response.response_builder:> Refined response: The new context does not provide any additional information that would require a refinement of the original answer. The original answer remains accurate and complete.

Refined response: The new context does not provide any additional information that would require a refinement of the original answer. The original answer remains accurate and complete.

INFO:llama_index.token_counter.token_counter:> [query] Total LLM token usage: 9426 tokens

'Tree Index':
{"query_str": user_question,
# "mode": "S",
"service_context": service_context,
"verbose": True
# "use_async": True
},
'Simple Vector Index':
{"query_str": user_question,
"mode": "default",
"response_mode": "tree_summarize",
"similarity_top_k": 5,
"service_context": service_context,
"verbose": True
# "use_async": True
},

i want to extract the docstore from the index into a table how can i do that?

Hi, im using a list index but u need to speed up the processing time. My use case doesn’t allow for me to risk missing important information so a list index is required there a way to case doesn’t allow for me to risk missing important information so a list index is required. Is there a way to maybe with trees summarize send out multiple API calls at once?

@Logan M is this true?

@kapa.ai the LLMRerank node post processor isn't working for me, should i be adding that to the response synthesizer, the retriever or the query engine?

just wanted to give a huge shout out to @Logan M , different kind of message than you may be used to in this chat haha but honestly thank you for all your attentiveness, quick responses, and helpfulness. also congrats and big thanks for the big update today with native hugging face support !!

hi team, when loading an index with storage context after storage context already contains the docstore why does it need to be called with nodes? aren't the nodes already in the docstore and subsequently in the storage_context?

from llama_index.docstore import MongoDocumentStore
from llama_index.node_parser import SimpleNodeParser

create parser and parse document into nodes

parser = SimpleNodeParser()
nodes = parser.get_nodes_from_documents(documents)

create (or load) docstore and add nodes

docstore = MongoDocumentStore.from_uri(uri="")
docstore.add_documents(nodes)

create storage context

storage_context = StorageContext.from_defaults(docstore=docstore)

build index

index = GPTVectorStoreIndex(nodes, storage_context=storage_context)

hi team when using LLMMultiSelector i'm getting a weird error

File "/Users/sheresaidon/virtualenvs/bright-black-ai-chat-template/lib/python3.10/site-packages/llama_index/selectors/types.py", line 19, in ind
raise ValueError(
ValueError: There are 3 selections, please use .inds.

i have defined 3 tools as below and it works fine with SingleSelector
query_engine = RouterQueryEngine(
selector=LLMMultiSelector.from_defaults(
service_context=service_context),
query_engine_tools=[
list_tool,
vector_tool,
empty_tool
]

)