LlamaCPP

When it first loads the model, there should be a ton of prints about allocating to the gpu or something

Basically you need to make sure you installed with gpu support, and set num_gpu_layers to something other than zero