Find answers from the community

Updated 8 months ago

Template

Plain Text
2 frames
/usr/local/lib/python3.10/dist-packages/text_generation/client.py in chat(self, messages, repetition_penalty, frequency_penalty, logit_bias, logprobs, top_logprobs, max_tokens, n, presence_penalty, stream, seed, temperature, top_p, tools, tool_choice)
    152             payload = resp.json()
    153             if resp.status_code != 200:
--> 154                 raise parse_error(resp.status_code, payload)
    155             return ChatComplete(**payload)
    156         else:

UnknownError: Template error: template not found

tgi help
L
d
17 comments
Seems like the model you are using doesn't have a chat template in huggingface
but the completin works?
its a fine-tune llama
can we a apply one or naw
Completion doesn't use a chat template

The chat template is used to transform message dicts into model prompts.

Specifically, it uses tokenizer.apply_chat_template(..)
Only way to fix it is to ensure the tokenizer has that method πŸ˜…
its llama 32 k lol
Maybe the transformers version isn't latest?
wait so completion works but retriver no works?
Plain Text
service_context, text_qa_template, refine_template, summary_template, simple_template, response_mode, callback_manager, use_async, streaming, structured_answer_filtering, output_cls, program_factory, verbose)
     71             or Settings._prompt_helper
     72             or PromptHelper.from_llm_metadata(
---> 73                 llm.metadata,
     74             )
     75         )

/usr/local/lib/python3.10/dist-packages/llama_index/llms/huggingface/base.py in metadata(self)
    832     @property
    833     def metadata(self) -> LLMMetadata:
--> 834         return LLMMetadata(
    835             context_window=self.context_window,
    836             num_output=self.max_tokens,

/usr/local/lib/python3.10/dist-packages/pydantic/v1/main.py in __init__(__pydantic_self__, **data)
    339         values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340         if validation_error:
--> 341             raise validation_error
    342         try:
    343             object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for LLMMetadata
model_name
  none is not an allowed value (type=type_error.none.not_allowed)
 
That looks like am issue with how the llm object was created πŸ˜… missing a model name
ve been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
WARNING:llama_index.llms.huggingface.base:TGI client has no function call support: ("'text-generation-inference' version ", 'incompatible with function call: 1.4.5. ', 'Function call support was added in v2.0.1')
TGI client has no function call support: ("'text-generation-inference' version ", 'incompatible with function call: 1.4.5. ', 'Function call support was added in v2.0.1'
its my fault lol
should just use gpt4
Add a reply
Sign up and join the conversation on Discord