LLAVA Bugs

I have used all the necessary commands in Google collab @Haotian
!git clone https://github.com/ggerganov/llama.cpp.git
%cd llama.cpp
!make
!Wget https://huggingface.co/mys/ggml_llava-v1.5-7b/blob/main/ggml-model-f16.gguf
!./llava
let me know if i was missing something

@bigbeard here u go

got it!

@Haotian looks like there's an unofficial version here? https://replicate.com/yorickvp/llava-13b

cc @Gao_AI

sorry for jumping into this late. @bigbeard yes we are serving llava thru replicate now. is it working now for you?

No worries! I'll loop back and mess around with it. From what I remember it was working. Thank you!

@Haotian Wanted to give some feedback. Seems like the model doesn't handle 0 Temperature well. Also, when scanning some images with a lot of text (to perform an OCR image to text task) it seems to get "stuck" in the first few lines especially for some image scanes of a paper document

@bigbeard yeah I found llava is not very good at OCR kind of job. GPT4v is better than Llava in terms of this task. You could try gpt4v or try to feed Llava more comprehensive prompt

Is it also designed to work at Temperature > 0? It seems to only work when I set it greater than 0.01 or something