Search results

I think you are right, there's a difference between textual search vs. Semantic search.

Rather than searching for exact strings, I think it makes sense to search for the same "ideas"... if that makes sense

YEs makes sense for me as well.
But what if an user asks for a summary? EG: "can you summarize the doc xyz? Specifically the paragraph 123."

In this way the AI would summarize the doc xyw or it would respond that it can't find the document requested. I mean, it's so disappointing. A partial solution would be to include the "exclude" and "required" keywords...

But my point is: is it normal such a behaviour? Are you experiencing the same kind of results? If yes, I don't know what i did in these 2 months, but at least I will use the partial solution...

For summaries, it's better to use a list or tree index.

And actually, if you are using langchain, this makes a lot of sense to make different indexes for different use cases.

I.e. you can make a tool for each index, with appropriate descriptions for "summarizing" and "search" and any other use case.

Then, Langchain will pick the right index to use depending on the user query

Maybe have a read on this page, for different use cases https://gpt-index.readthedocs.io/en/latest/use_cases/queries.html

It's really a wonderful idea. I didn't think to that!