togethercomputer/GPT-NeoXT-Chat-Base-20B...

Just tested out the Chatbot they give with it and yeah this is quite decent.

Yea! We can load this locally and use it with llama index potentially

20B fits on 3060 <?

12GB vram right? Google tells me you probably need a 3090 minimum with 8bit quantization 💸

I suspect the community will be working on other methods to load this into smaller GPUs (similar to the recent progress witg llama), so keep on eye on github and whatnot. Still very fresh

It's pretty good!

I have a 3090 actually, maybe I have to test this out if there is an easy way 🙂

Just in the last few hours I am starting to see more and more people testing llama. Getting 30B working on mini mac books at 8bit.

Not an expert on model architecture… is it possible to parallelize across GPUs?