Find answers from the community

Updated 2 years ago

i was a bit overwhelmed with all the

i was a bit overwhelmed with all the different indexes. Is there some documentation on when i want to use a certain index over the other?

My usecase is I have 1000s of tweets collected and I want to ask the llm to find ones with certain subjects or summarize tweets based on a certain search criteria.

Edit: taking a look at https://gpt-index.readthedocs.io/en/latest/guides/index_guide.html

5 comments

jjerryjliu0

yep! please take a look, lmk if any questions

MMikko

My 2cnts, if you are working on a single document that is relatively short, like a 2 page pdf, start with the simplest list index. Then move to the simple vector index and you'll get a feeling how the results change. The list index won't scale to a large corpus (or it will but cost is big), if applied naively over your data.

jjerryjliu0

yeah i'd recommend list index for smaller docs or simple vector index for anything beyond that to start with

aaioverdrivefriend

Is there a reason why we wouldn't use a keyword index versus a vector index?

jjerryjliu0

some of this can be for you to experiment with! a keyword index looks up content based on keywords, whereas a vector index looks up stuff based on embedding similarity

Add a reply