Find answers from the community

Updated 2 months ago

Privacy

Yea. I personally don't think its an issue but employer is paranoid about that stuff (its understandable)
L
B
5 comments
Yea definitely understandable!

You can use llama index with local models, but tbh open source stuff only works well for basic use cases.

Stuff like agents or more complex query engines may not work well πŸ˜…
Plus then you need the resources to run the local stuff πŸ€”
yea... i was gonna say probably cannot use a cheap aws instance for that
whats the best local model for basic stuff?
Probably something like the 7B falcon model it seems like. I usually check here

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

Anything above 7 to 13B and the requirements start to get a little complex. Although the 4bit quantization helps a lot
Add a reply
Sign up and join the conversation on Discord