Find answers from the community

Updated last year

Hey team

Hey team,
I am trying to use DocumentSummaryIndex and experiencing a very slow response time (up to 1 minute). I noticed that in the index, which I saved locally, the "embedding" field is empty. How can I use this field or improve the performance of the Document Summary index? Creation of index and inference below.


Thanks!
L
V
9 comments
The embeddings are calculated on the fly -- this is not really too efficient I know. There is a PR to improve this somewhat soon

That's about the only reason I can think of that it's slow. Is it just the first query that is slow? How many documents do you have?
Every query takes 40-60 seconds, where Retrieve takes most of the time.
400 documents (it`s webpages), but content is really small: two three-text paragraphs
@Logan M Can you tell me please, is the expected behaviour for such amount of documents?
no thats not expected, as I explained above about the efficiency, the embeddings are being calculated on every query. With 400 documents, that will be slow

This PR will fix it once it's merged
https://github.com/jerryjliu/llama_index/pull/7815
I applied changes from that pull request, and indeed, the answers were now given within 10 seconds. So, I am waiting for it once it is merged. πŸ™‚
I need to poke the guy to get it wrapped up haha
if you run into any issues on that branch in the meantime, let me know!
Add a reply
Sign up and join the conversation on Discord