Find answers from the community

Updated 2 years ago

i m getting a divide by zero from this

i'm getting a divide by zero from this line in prompt_helper.py:
Plain Text
        result = (
            self.max_input_size - num_prompt_tokens - self.num_output
        ) // num_chunks

i'm trying to load a GPTTreeIndex from persisted storage. not sure if related, but looks like it might be trying to load a nonexistent vectorindex. this is the line that leads me to think maybe that's what's going on:
Plain Text
INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 0 tokens

anyone got any ideas?
d
K
11 comments
Hey @Kavin, let me try to help debug. Is there a way to easily reproduce this issue?
yeah lemme share code one sec
Plain Text
import os
from llama_index import GPTTreeIndex, SimpleDirectoryReader, LLMPredictor, ServiceContext, StorageContext, PromptHelper
from llama_index import load_index_from_storage
from langchain.chat_models import ChatOpenAI
from langchain import OpenAI

# Ingest a specified directory of text files
directory_path = "resources/stories/single_story_test"  # Replace with your directory of text files
index = None

# Set maximum input size
max_input_size = 1024
# Set number of output tokens
num_output = 256
# Set maximum chunk overlap
max_chunk_overlap = 20

prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)

# Define LLM
llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-4", request_timeout=120))
service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)

storage_dir = "./.storage"
if os.path.isdir(storage_dir):
    if not os.path.isfile(os.path.join(storage_dir, "index_store.json")):
        raise ValueError(f"Directory {storage_dir} exists but could not find index_store.json inside.")
    print("Index already exists. Loading from persisted values.")
    storage_context = StorageContext.from_defaults(persist_dir=storage_dir)
    index = load_index_from_storage(storage_context, service_context=service_context)
else:
    print("Index does not exist. Creating from scratch.")
    documents = SimpleDirectoryReader(directory_path).load_data()

    # Index and summarize using GPTTreeIndex
    index = GPTTreeIndex.from_documents(documents, service_context=service_context)
    index.storage_context.persist(persist_dir=storage_dir)

# Query the index to get the summaries
query_str = "Summarize the plot of this story into numbered bullets, each representing approximatly 500 words"
query_engine = index.as_query_engine(response_mode="tree_summarize")
response = query_engine.query(query_str)

print(response)
llama-index==0.6.0a5
that's the text
i wonder if something is goofy with my system though because i tried copying the code from this example notebook and i got other problems - https://github.com/jerryjliu/llama_index/blob/main/examples/gatsby/TestGatsby.ipynb
Plain Text
import logging
import sys
import os
from llama_index import GPTTreeIndex, SimpleDirectoryReader

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

documents = SimpleDirectoryReader('resources/single_story_test').load_data()
new_index = GPTTreeIndex.from_documents(documents)

query_engine = new_index.as_query_engine()
response = query_engine.query("what is the name of the protagonist?")
print(response)
Plain Text
(venv) kavins-mbp-2:storygen kavinstewart$ python3 alt_summarize.py 
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens
> [build_index_from_nodes] Total LLM token usage: 0 tokens
INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 0 tokens
> [build_index_from_nodes] Total embedding token usage: 0 tokens
INFO:llama_index.token_counter.token_counter:> [retrieve] Total LLM token usage: 0 tokens
> [retrieve] Total LLM token usage: 0 tokens
INFO:llama_index.token_counter.token_counter:> [retrieve] Total embedding token usage: 0 tokens
> [retrieve] Total embedding token usage: 0 tokens
INFO:llama_index.token_counter.token_counter:> [get_response] Total LLM token usage: 0 tokens
> [get_response] Total LLM token usage: 0 tokens
INFO:llama_index.token_counter.token_counter:> [get_response] Total embedding token usage: 0 tokens
> [get_response] Total embedding token usage: 0 tokens
None
ok still debugging but here's what i've found so far... the problem seems to happen in select_leaf_retriever.py in the _retrieve_level function
here's the variable values:
Attachment
image.png
cur_node_list contains values but for some reason when it runs this code to create selected_nodes it ends up empty. not sure why yet:
Plain Text
            selected_nodes = self._select_nodes(
                cur_node_list,
                query_bundle,
                level=level,
            )
Add a reply
Sign up and join the conversation on Discord