Llama

Llama index supports and llm, but it's up to you to write the code for passing the text to the model and returning the newly generated tokens.

Here's a small example with googles FLAN and huggingface

https://github.com/jerryjliu/llama_index/issues/544

I think I need to add this to the docs lol 😆

Hi

I was about to ask the same thing!

One more vote to add it to the docs then lol

but also, is there a recommended local embedding model for generating indexes?

or do we have to use "text-embedding-ada-002"

It will probably take some experimenting. Sentence transformers is a good start: https://huggingface.co/sentence-transformers/all-mpnet-base-v2

Some docs on customizing embeddings here: https://gpt-index.readthedocs.io/en/latest/how_to/embeddings.html#custom-embeddings

thanks, thats defo something for the docs! I see Alpaca 30B was released on HuggingFace recently; with some training, it would be my go-to for this stuff

it should run nicely in Int4 on a consumer GPU

Does it make sense to move from embeddings-ada-002 to other models, like those proposed by langchain? Do they perform better/faster?

What is langchain proposing? I haven't seen that 👀

Personally, Ada is pretty good, the input size is huge (8196), and is super cheap.

Lots of companies (in Europe) are not comfortable sending their data to 3rd parties

having a fully local system has a lot of value