Find answers from the community

Updated 6 months ago

I am using llama index to build a local

At a glance

The community member is using LlamaIndex to build a local chat application using data from MySQL, and they are looking for a small model (around 1GB or less) to use instead of the OpenAI API. In the comments, another community member suggests that a model that small would not work well, as large language models typically require at least 13 billion parameters to be useful. The community member then thanks the other for the suggestion and mentions they will look into a project called openssm, which was provided as a potential alternative.

Useful resources
I am using llama index to build a local chat of data from mysql.
i don't want to use openAI api and wanna build a local chat - small model around <=1GB can you help suggest 1 model please.
Thanks
L
C
2 comments
A <1GB model will never work well tbh. It's pretty much impossible.

I've found an LLM needs at least 13 billion parameters to be useful. 1GB is like 1B parameters or less πŸ˜…

The closest thing I've seen is this project, but I haven't tried it out yet
https://github.com/aitomatic/openssm
thanks for pointing the direction. I'll take a look.
Add a reply
Sign up and join the conversation on Discord