I m asking this question

I'm asking this question:

give me as much detail about these documents as you can

and I'm getting this response:

Plain Text

The new context information provided is a list of numerical values, which are likely the embeddings_dict mentioned in the original answer. These values may be used to represent the document in a numerical format for machine learning or natural language processing purposes. However, this information does not provide any additional details about the document store or the specific document mentioned in the original answer. Therefore, the original answer remains the same.

heres my code:

Plain Text

def queryIndex(indexes, query):
    
    catsIndex = Document(text=json.dumps(indexes[0]))
    dogsIndex = Document(text=json.dumps(indexes[1]))
    
    combined = GPTSimpleVectorIndex([catsIndex, dogsIndex])

    llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo", max_tokens=250))
    return combined.query(query, mode="default", llm_predictor=llm_predictor)

when I pass in indexes it's a list of dicts.

anyone have any idea what I'm doing wrong here?

18 comments

LLogan M

what are indexes made from? Are they json files generated by llama_index?

iitsgeorgep

yep. A user uploads a file, then I do:

Plain Text

def buildIndex(text):
    document = Document(text=text)
    index = GPTSimpleVectorIndex([])
    index.insert(document)
    return index.save_to_string()

Then save that to the db, then later when they query, I do the above

LLogan M

Cool! So since the index is from a save, then you'll want to load it like this instead of using the raw json text

Plain Text

catsIndex = GPTSimpleVectorIndex.load_from_string(indexes[0])
catsIndex.set_text("This contains information about cats.")

dogsIndex = GPTSimpleVectorIndex.load_from_string(indexes[1])
dogsIndex.set_text("This contains information about dogs.")

combined = GPTListIndex([catsIndex, dogsIndex])

llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo", max_tokens=250))
return combined.query(query, mode="recursive", llm_predictor=llm_predictor)

iitsgeorgep

Thanks! I'm definitely getting closer. At least it's not erroring out any more

iitsgeorgep

here's what I get

Attachment

Screen_Shot_2023-03-18_at_7.08.48_AM.png

iitsgeorgep

At the moment, in my app, each chat is independent (right now)... It doesn't take into account the previous message yet.

So I don't know why I'm getting these answers. That first answer, "The original answer is not useful..." makes no sense.. there is no "original answer", that was my first query to the file

LLogan M

So, under the hood in the code, it's actually making multiple calls to the LLM and refining an answer.

So since we have two vector indexes inside a list index, one query is making two calls to the LLM

ChatGPT has some issues with this process sometimes and doesn't follow the internal instructions (you can see how stubborn it is haha)

Are you using the latest version of llama index? This should have been improved in recent versions 🙃

Another option is to add response_mode="compact" as an argument to the query call. This will try to reduce how many calls to the LLM that is made

iitsgeorgep

Ah ok. To make it one call to the LLM would I combine the indexes manually and pass that “catsAndDogsIndex” into GPTlistIndex?

LLogan M

Almost! To try and ensure one LLM call, you'll need everything in a single vector index actually (the default is similarity_top_k=1 for vector indexes, which means it fetches the closest matching node to the query)

So when you construction your index, you might need to either call insert() for each document (the actually document text, rather than the index), or pass all documents in during the initial construction index = GPTSimpleVectorIndex([cat_document, dog_document...])

iitsgeorgep

Interesting response:

iitsgeorgep

Attachment

Screen_Shot_2023-03-18_at_6.33.53_PM.png

iitsgeorgep

Plain Text

document1 = Document(text="I like dogs")
document2 = Document(text="I like cats")
index = GPTSimpleVectorIndex([document1, document2])
llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=.25, model_name="gpt-3.5-turbo", max_tokens=500))
return index.query(query, mode="default", llm_predictor=llm_predictor, response_mode="compact")

LLogan M

Lol yea that's kinda weird. Pretty hallucinate-y

You have the latest llama index version right? pip install --upgrade llama_index langchain

I might also stick with temperature=0

iitsgeorgep

Is there a lower level way to debug to see all the calls to the model?

LLogan M

set the level to logging.DEBUG

Here's an example at the top of this notebook (but it's setting it to INFO) https://github.com/jerryjliu/llama_index/blob/main/examples/vector_indices/SimpleIndexDemo-ChatGPT.ipynb

iitsgeorgep

@Sahil @Rashika

iitsgeorgep

@Markos

MMarkos

I am trying to generate response with query transform and I get a result like this.

"The original answer remains relevant and does not need to be refined.......". I have the latest version of langchain and llamaindex.

Add a reply

Find answers from the community

I m asking this question