LlamaIndex

Log inLog into community

Find answers from the community

Updated 2 years ago

It s pretty important to be able to

It s pretty important to be able to

At a glance

·

It's pretty important to be able to answer a question like 'When are the tutorials' or 'What tutorial is Today' and such.

Generally the #1 issue I'm having with the index is not pulling the context when it is there in the docs. I am A-Okay spending more tokens to feed more context in, but am unsure how to do that (if that is the solution).

Using the CONTEXT chat engine.

L

i

32 comments

In general, retrieval is hard lol

I'm not sure what your setup looks like, but once you start having a lot of data in a single index, retrieval gets harder and harder.

There are a few suggestions that I can give to help improve results

Split indexes/retrievers into topics, and use a router to retrieve from the proper index https://gpt-index.readthedocs.io/en/latest/examples/query_engine/RetrieverRouterQueryEngine.html
Use more fine-grained embeddings with smaller chunks. In this example, we embed each sentence, and then at retrieval time, post-process the nodes to show the wider context https://gpt-index.readthedocs.io/en/stable/examples/node_postprocessor/MetadataReplacementDemo.html
Increase the top k by decent amount (10? 15? 20?) and use a re-ranker https://gpt-index.readthedocs.io/en/stable/core_modules/query_modules/node_postprocessors/modules.html#sentencetransformerrerank
If you are using a vector db that supports hybrid search, that can also help (especially when also combined with a re-ranker)

In general, retrieval is hard lol

For sure. In terms of data size, no more than 3 or 4 pages of double spaced info, so not a lot. The setup is just a slightly customized VectorStoreIndex and we are using all defaults with the context engine.

Will check out the suggestions... I just needed a jumping off point 😄
The docs are great if you know what you are looking for haha

Thanks

heh yea, you kind of just need to spend time in the docs to know whats possible 😆

3-4 pages sounds pretty manageable though 🤔 Maybe a simple top k increase + re-ranking will do the trick here, so that you have a better chance of pulling the correct info

heh yea, you kind of just need to spend time in the docs to know whats possible 😆

The pill no programmer wants to swallow but knows they have to

Top K increase would be in the query engine, or the index itself?

Will look at all those suggestions, tysm btw

should be something like this index.as_query_engine(similarity_top_k=2) (default is 2)

Cool, will set to 10^3

Jokes

Thanks

Hey Logan,

Regarding the suggestion to use sentence splitting. I implemented it, and while it's great at specific sentences, it now has a lot of difficulty with more 'overall' questions

Any off-the-cuff suggestions?

Attachment

Screen_Shot_2023-08-24_at_3.43.42_PM.png

For example, "what are the evaluations" returns nothing useful, but "how much is activity 9 worth" targets correctly

Hmmm, interesting observation.

Just as a refresher, this is using the SentenceWindowNodeParser?

Or just the sentence splitter inside the SimpleNodeParser?

Sentence window passed

Parser

Let me debug the context received

Maybe it's a matter of adjusting the window size? 🤔 the default is 3 right now

Plain Text

Context information is below.\n--------------------\nThe assignment will comprise the following elements.\n\nAssignments will be handed in electronically through eClass (not email).\n\nPre-class quizzes ensure that readings were completed, tutorial presentations and discussions focus on the readings, lectures provide contextual and conceptual knowledge that helps to understand the readings and assignments are conceived in a way to delve deeper into the readings.. No grade per se is assigned to participation.\n--------------------\n

🤔 interesting

Plain Text

What assignments are there?

Tough situation...

oh, my top k was 3

Hmmm, are you also using the Metadata replacement node-postprocessor.

You could also crank the top k yea haha

yes

looks better now

top k is 5, and i changed the "sentence" separator to be a newline OR 250 characters

then set window size to 9

averages ~400 tokens

Nice!

Add a reply

Sign up and join the conversation on Discord