Find answers from the community

Updated 7 months ago

any idea when the new gemma2 will be

At a glance

The community member is experiencing an issue with the "gemma2" model, where they are encountering an error when trying to load it. They are eager to test out the new model. In the comments, another community member suggests that the model is being downloaded from Hugging Face, and provides a link to the specific model page. The community member then shares the code they are using to load the model, which includes some configuration options.

Useful resources

DDigital Rally

any idea when the new gemma2 will be working?

Plain Text

llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma2'

cant wait to test it out 😍

4 comments

WWhiteFang_Jr

Are you downloading it from HF?

DDigital Rally

yes, from bartowski

DDigital Rally

https://huggingface.co/bartowski/gemma-2-9b-it-GGUF

DDigital Rally

im using it with llama cpp

Plain Text

llm = LlamaCPP(
    model_path=model_path,
    model_kwargs={
        "n_gpu_layers": 50,
        "grammar_path":"/home/_LLM/llama.cpp/grammars/json.gbnf",
        "repeat_penalty": 0.0},
    temperature=0.0,
    max_new_tokens=max_output_tokens,
    context_window=context_window,
    verbose=True
)

Add a reply