Template

Plain Text

2 frames
/usr/local/lib/python3.10/dist-packages/text_generation/client.py in chat(self, messages, repetition_penalty, frequency_penalty, logit_bias, logprobs, top_logprobs, max_tokens, n, presence_penalty, stream, seed, temperature, top_p, tools, tool_choice)
    152             payload = resp.json()
    153             if resp.status_code != 200:
--> 154                 raise parse_error(resp.status_code, payload)
    155             return ChatComplete(**payload)
    156         else:

UnknownError: Template error: template not found

tgi help

17 comments

LLogan M

Seems like the model you are using doesn't have a chat template in huggingface

ddrewskidang

but the completin works?

ddrewskidang

its a fine-tune llama

ddrewskidang

can we a apply one or naw

LLogan M

Completion doesn't use a chat template

The chat template is used to transform message dicts into model prompts.

Specifically, it uses tokenizer.apply_chat_template(..)

LLogan M

Only way to fix it is to ensure the tokenizer has that method 😅

awww man

its llama 32 k lol

probably

why

Maybe the transformers version isn't latest?

ddrewskidang

wait so completion works but retriver no works?

Plain Text

service_context, text_qa_template, refine_template, summary_template, simple_template, response_mode, callback_manager, use_async, streaming, structured_answer_filtering, output_cls, program_factory, verbose)
     71             or Settings._prompt_helper
     72             or PromptHelper.from_llm_metadata(
---> 73                 llm.metadata,
     74             )
     75         )

/usr/local/lib/python3.10/dist-packages/llama_index/llms/huggingface/base.py in metadata(self)
    832     @property
    833     def metadata(self) -> LLMMetadata:
--> 834         return LLMMetadata(
    835             context_window=self.context_window,
    836             num_output=self.max_tokens,

/usr/local/lib/python3.10/dist-packages/pydantic/v1/main.py in __init__(__pydantic_self__, **data)
    339         values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340         if validation_error:
--> 341             raise validation_error
    342         try:
    343             object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for LLMMetadata
model_name
  none is not an allowed value (type=type_error.none.not_allowed)

ddrewskidang

okay i try

LLogan M

That looks like am issue with how the llm object was created 😅 missing a model name

ddrewskidang

ve been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
WARNING:llama_index.llms.huggingface.base:TGI client has no function call support: ("'text-generation-inference' version ", 'incompatible with function call: 1.4.5. ', 'Function call support was added in v2.0.1')
TGI client has no function call support: ("'text-generation-inference' version ", 'incompatible with function call: 1.4.5. ', 'Function call support was added in v2.0.1'

ddrewskidang

its my fault lol

ddrewskidang

should just use gpt4

Add a reply

Find answers from the community

Template