Llama index is quite a bit more customizable and flexible.
Llama index uses openAI models to answer questions, but you can also use any LLM you want.
Plus Llama index has a lot of powerful index structures, and let's you easily manage where your data lives. You can also use llama index as a custom tool in langchain, which opens up even more use cases
Thx!. But then what algorithm does Llama index use for query? I saw some examples that Llama can query some information from the document without OpenAI or at least without providing a OpenAI API key.
The underlying algorithm remains the same with or without openai. All the changes is which model you are sending text to to answer questions 💪
For a vector index, the basic algorithm is to generate embeddings for every node. Then at query time, generate an embedding for the query text and retrieve the top k matching nodes using cosine similarity. Then, an answer is synthesized using the top k chunks as context.
If all the text does not fit into one LLM call, the answer is refined across a few calls, so that the LLM ends up reading all the context.
Both the LLM and Embedding Model can be set to be anything you want (OpenAI, cohere, or something from huggingface!)
