LlamaIndex
Log in

Find answers from the community

Updated 3 months ago

Questions

ccaptam_morgan
·
I have three questions:

  1. Are there use cases where decomposable graph makes more sense than subquestion query? I feel like sub-question can handle everything graph does?
  1. Maybe this has to do with node post-processing. Is there a dynamic way to set similarity_top_k so I always use the maximum # that can fit inside the context window?
  1. Does llama index offer any smart chunking algorithms? For example, instead of a fix length cutoff, can I do it by paragraphs or contextual topics?
L
1 comment
LLogan M
  1. Yea the graphs are basically deprecated. Other query engines like sub question work much better imo
  1. Hmm, yea I think this would be a post processor. Set the top k super high, and then limit it by counting tokens?
  1. Mmm not exactly. But there are some more complex algorithms
Sentence Window:
https://gpt-index.readthedocs.io/en/stable/examples/node_postprocessor/MetadataReplacementDemo.html

Hierarchical chunk merging:
https://gpt-index.readthedocs.io/en/latest/examples/retrievers/auto_merging_retriever.html

Automatic metadata extraction:
https://gpt-index.readthedocs.io/en/stable/examples/metadata_extraction/MetadataExtraction_LLMSurvey.html
Add a reply
Sign up and join the conversation on Discord