Hi guys, I ma having issues with my code

Oh no, I see the LlamaToolKit in there 😅 Quite deprecated/unmaintained, I'm surprised it works still

So you are making an index per document using the sentence window stuff, then putting that into a sub question query engine

How many index/documents do you have?

No, the LlamaToolKit doesnt work, I was just testing the memory module. It actually sucks. 🙂

So its about 14 files

I think the main issue is that every tool description is the exact same, minus a tiny piece (the doc name)

Maybe generating a summary on the fly (and caching that somewhere) and using that as the description would be better?

I will try that.

Thanks.

Hi logan I implemented this but I am still having the same issue. It find the answer but doesnt notice it

Updated code

Just a look at the output

out of curiosity, which LLM are you using?

Llama2?

You'll notice it queried nearly every document it looks like :PSadge: Thats a lot of returned context for an LLM to sift through

ideally it would only query 1-5 at most 🤔

I wonder if using a smarter LLM for the question generator would help

like gpt-4 or gpt-4-turbo

if openai is not an option, then you'll need to tackle this another way -- likely something that relies on embeddings, like a retriever router? https://docs.llamaindex.ai/en/stable/examples/query_engine/RetrieverRouterQueryEngine.html

I am using 3.5 turbo

I will have a look at that

Thanks again

Hi @Logan M logan, I am still struggling to get specific information out of the LLM. I am really not sure how to improve upon this,

I have implements retriever routers with no success