Reranker

Its not really a bug. It sounds like the llm is just not outputting proper text to parse for reranking to work

Ngl I would never use an llm to rerank in production -- slow, expensive, unreliable

Thank you for that comment. I might open an issue about it, I am mostly exploring anyway. I will let you know if I do.

Just to check, though: if your ngl comment stands, why do you have it as an example for people to use in LlamaIndex? I would expect documentation to show good practices. Not "slow, expensive, unreliable" ones. (Or at least say it shouldn't be used in practical applications!)

(also sorry for the slow reply, I got an e-mail notification about your reply yesterday for some reason...)

(And of course, I really like llamaindex, I think you are doing a great job overall. I just hate it when tutorial don't work for no obvious reason.)

I think its an example of something that could scale better someday.

Anything that relies on parsing an LLM output is not going to be 100% reliable -- thats just the nature of LLM apps these days

thats my hot take, anyways haha

Yeah, I can see that, I noticed it particularly when using the MultiStepQueryEngine functionality (which is very cool otherwise).