Retrieval

At a glance

The post shows some code for setting up a vector store and indexing documents. Community members in the comments discuss the performance of this setup, noting that the chunk size and overlap seem suboptimal. They suggest using a larger chunk size (e.g., 512) and increasing the top k retrieved nodes. The community members also note that the data being indexed (4.78 MB across 4 documents) may be too large, and that the presence of tables in the data could be causing issues with storage and retrieval. They provide suggestions for improving performance, such as using a reranker with a top-n of 3 or 4.

AAmitKhandey

but this work very well :vector_store = self.setup_chroma(collectionName)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

documents = SimpleDirectoryReader(
input_files = files
).load_data()

node_parser = SentenceSplitter(chunk_size=180, chunk_overlap=80)
nodes = node_parser.get_nodes_from_documents(documents)
index = VectorStoreIndex(nodes, storage_context=storage_context)

15 comments

LLogan M

Why is your chunk size so small? (And overlap is very large here compared to chunk size)

LLogan M

If you aren't changing the top k, it's only retrieving the top 2 nodes -- and both those nodes are tiny

LLogan M

So retrieval will probably not be good with these settings

AAmitKhandey

What should be ideal size of chunk and chunk_overlap

AAmitKhandey

I am using gpt-3.5-turbo AND text-embedding-ada-002

LLogan M

The default is a good choice (1024)

512 is also not bad.

Chunk overlap depends on the chunk size generally. I usually go with 20, but it's less important

AAmitKhandey

how can I improve performance?

AAmitKhandey

RAG is unable to give proper answer to questions I did chunking before storing in chroma

AAmitKhandey

what else can be done?

LLogan M

Did you change the chunk size? How much data are you indexing?

LLogan M

what kinds of questions are you asking? You can debug retrieval somehwat by checking the source nodes

Plain Text

response = quer_engine.query("...")
for node in response.source_nodes:
  print(node.text)

AAmitKhandey

Yes I did change the size. the size of data is 4.78 MB which is 4 documents.

AAmitKhandey

Most of data are in form of tables in word document and excel

AAmitKhandey

I think it is having trouble storing and retrieving of data in table

LLogan M

4.78 MB is a looot of text 😅 I would suggest
a) using a chunk size of 512
b) increasing the top k -- probably 12?
c) using a reranker with top-n = 3 or 4

Add a reply

Find answers from the community

Retrieval