YannZeRookie

Hello everyone I made good progress with

Hello everyone! I made good progress with my AI app: using a Qdrant vector storage and OpenAI, I can query a French book and it works great 🙂
Now, what about English? When I ask a question in English, it answers properly in English using the context of the book. It's magic. Of course, proper names remain in French, which makes sense.
It happens that I have a French -> English glossary for these names. I wrote a test OpenAI prompt in which I give the first English/French answer and a table of the few relevant glossary entries, and presto, it rewrites the text by replacing the French names by their English equivalent. Very nice. So, where is my problem?
My problem is to find these "relevant glossary entries". I thought about building a separate Vector Store with the Glossary, and perform a query() with the first English/French answer, but the list of Nodes I get back is very far from being relevant (my glossary has one Node per entry). Even when I increase greatly the number of returned Nodes, I don't get relevant entries. I even get entries that have words that are nowhere in the text. Weird.
How can I narrow the search?

1 comment

YYannZeRookie

Nodes

Can anyone tell me how LlamaIndex sends the content of the Nodes to the LLM? Does it do so by filling the {context_str} variable in the retriever prompt template? I searched in the code and did not find it

4 comments

YYannZeRookie

Hello everyone

Hello everyone!
First post on this Discord server - I hope I am using the right channel.
I have been suffering for 2 months with LangChain until I discovered LlamaIndex. What a better documentation and an easier use!
My question is related to VectorStore Indexes, and especially Chroma. I processed a large book (362k words) and tested various VectorStore solutions.

the default index provided in the LlamaIndex tutorials give decent results but is slow and does not seem to be designed for a production real-life app.
Using Pinecone gives me great results (scores at 0.83 and great responses from the LLM).
Chroma was my next test, since it is free and I have my own server. But I get terrible results! The scores of retrieved Nodes are around 0.23 and (not surprisingly) the LLM gives me very bad results ("junk-in, junk out" effect I guess). I tried to play with various parameters, but did not have luck.

Code extract:

Plain Text

print("Building the Chroma VectorStore...")
db = chromadb.PersistentClient(path="./laelith_chroma")
chroma_collection = db.get_or_create_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
embed_model = OpenAIEmbedding(OpenAIEmbeddingMode.SIMILARITY_MODE)

service_context = ServiceContext.from_defaults(
  llm=llm,
  embed_model=embed_model,
  chunk_size=400, 
  chunk_overlap=20
  )
index = VectorStoreIndex.from_documents(
  documents=documents,
  storage_context=storage_context,
  service_context=service_context,
  show_progress=True
)

Any ideas?

8 comments

Find answers from the community

Hello everyone I made good progress with

Nodes

Hello everyone