Find answers from the community

Updated 2 months ago

Llama

I'm sure I'm might not be the only one asking this, but is there support for local install of Alpaca for Llamaindex? I hate spending money each time I run a query.
L
d
A
14 comments
Llama index supports and llm, but it's up to you to write the code for passing the text to the model and returning the newly generated tokens.

Here's a small example with googles FLAN and huggingface

https://github.com/jerryjliu/llama_index/issues/544
I think I need to add this to the docs lol ๐Ÿ˜†
I was about to ask the same thing!
One more vote to add it to the docs then lol
but also, is there a recommended local embedding model for generating indexes?
or do we have to use "text-embedding-ada-002"
It will probably take some experimenting. Sentence transformers is a good start: https://huggingface.co/sentence-transformers/all-mpnet-base-v2

Some docs on customizing embeddings here: https://gpt-index.readthedocs.io/en/latest/how_to/embeddings.html#custom-embeddings
thanks, thats defo something for the docs! I see Alpaca 30B was released on HuggingFace recently; with some training, it would be my go-to for this stuff
it should run nicely in Int4 on a consumer GPU
Does it make sense to move from embeddings-ada-002 to other models, like those proposed by langchain? Do they perform better/faster?
What is langchain proposing? I haven't seen that ๐Ÿ‘€

Personally, Ada is pretty good, the input size is huge (8196), and is super cheap.
Lots of companies (in Europe) are not comfortable sending their data to 3rd parties
having a fully local system has a lot of value
Add a reply
Sign up and join the conversation on Discord