any guess for this error using the gpu

seems like it is something to do with service_context = ServiceContext.from_defaults(chunk_size_limit=512, llm_predictor=hf_predictor, embed_model=embed_model)

no ?

Not an error, just a warning. Something to do with the tokenizer somewhere I think, but it's pretty benign (I think the chunk size being the same is just a coincidence)

well

Ok, stupid asf, i have only 6gb of vram

Ooo yea 6gb isn't going to cut it lol

15GB minimum for that camel model

yeah, so it depends on the model no ?

A100's, here I am !

Yea depends on the model, as well as some other tricks you use (some models support 8 bit loading, or loading in 16 bit, etc.)

A100 is a good choice lol

update on this one, using a A6000 with 48G of VRAM, i'm 20min in and i've got the same error and no response for the moment. How much time did it take you to load ? @Logan M

and doing nvidia-smi it seems that this doesnt use that much of my vram

i checked and it uses like 1gb/48gb

don't know why

with this code