Results

iicsy7867

Driving me nuts! I can't figure out why my embeddings and sources/documents returned by llama index are all of a sudden different. The results are bizzare!

I have several articles ingested. The two I will reference as an example is an article about cloud services within my company. The other an article referencing how to install matlab. I will call my company name (abbreviated) XYZ.

"How do I install matlab?" - Incorrect sources returned
"How do I install matlab? XYZ - Correct sources returned
"How do I install matlab? G" - Correct Sources returned
"How do I install matlab? Flux Capacitor" - Correct Sources returned
"How do I install matlab? How do I install matlab?" - Correct Sources returned

Similarly...

"Cloud Services" - Incorrect Sources returned
"Cloud Services Cloud Services" - Incorrect Sources returned
"Cloud Services. Cloud Services" - Incorrect Sources returned
"Cloud Services Cloud Services." - Incorrect Sources returned
"Cloud Services. Cloud Services." - Correct Sources returned

Driving. Me. Nuts.
Hope someone has a magical solution 😄

Using ollama and nomic-embed-text for embeddings. Using Llama_index via https://github.com/zylon-ai/private-gpt

And I should note I was using the tool happily before. Something has changed. I even tried loading code from known working code with the same result. I can't figure it out.

17 comments

LLogan M

Did you recently recreate your index? Or otherwise change anything on your end?

What version of llama-index packages do you have? pip freeze | grep llama-index

iicsy7867

I've wiped the database and restarted multiple times to test after having issues.

iicsy7867

Plain Text

llama-index-core==0.10.14.post1
llama-index-embeddings-huggingface==0.1.4
llama-index-embeddings-ollama==0.1.2
llama-index-embeddings-openai==0.1.6
llama-index-llms-ollama==0.1.2
llama-index-llms-openai==0.1.6
llama-index-llms-openai-like==0.1.3
llama-index-readers-file==0.1.6
llama-index-vector-stores-qdrant==0.1.3

LLogan M

hmm, hard to say tbh. I know once you start adding more and more data to an index, retrieval gets pretty hard, and you need stuff like reranking and query rewriting to get more accurate results

iicsy7867

I also noticed llama-index-vector-stores-qdrant looks pretty outdated

iicsy7867

0.2.8 gave me some errors with the collection_name method. But 0.2.1 seems to work. Going to see if that helps.

iicsy7867

I've uploaded a couple hundred small documents... not a ton.

iicsy7867

It's odd that it was working fine last week, something changed and its just making me feel a little insane.

LLogan M

0.2.8 is not outdated, but requires the hosted qdrant version to be 1.8 or higher

iicsy7867

llama-index-vector-stores-qdrant==0.1.3

iicsy7867

But... this seems accurate. What you said before. After ingesting my first short wave, my test works fine.

iicsy7867

After ingesting another large chunk of items, it all of a sudden cant find the correct articles.

iicsy7867

¯\_(ツ)_/¯

LLogan M

yea this is kind of what I alluded to earlier -- as you add more data, you have to increase the top-k, and use other retrieval strategies to improve results

LLogan M

Once you have enough data, naive top-k is not enough

iicsy7867

The project recently added repacker capabilities. I'll give that a shot

iicsy7867

Though I'm not sure why all of a sudden I'm needing it when I haven't hadded more info.

Add a reply

Find answers from the community

Results