this is my query wrapper: query_wrapper

At a glance

The community member is encountering a ValidationError when initializing a HuggingFaceLLM object, specifically related to the query_wrapper_prompt parameter. The error message indicates that the parameter is expecting a string, but it's receiving an instance of SimpleInputPrompt. The comments suggest that the community member should try using the query_wrapper_prompt directly as a string when passing it to the HuggingFaceLLM constructor, instead of using an instance of SimpleInputPrompt. The comments also recommend checking the LlamaIndex documentation for more information on how to use the query_wrapper_prompt parameter.

FFtm

@kapa.ai this is my query wrapper: query_wrapper_prompt = SimpleInputPrompt("<|USER|>{query_str}<|ASSISTANT|>")

but iam getting this error:
-----------------------
ValidationError Traceback (most recent call last)
Cell In[1], line 25
23 qa_template = Prompt(template)
24 # Initialize HuggingFaceLLM
---> 25 llm = HuggingFaceLLM(
26 context_window=1028,
27 max_new_tokens=256,
28 generate_kwargs={"temperature": 0.7, "do_sample": False},
29 system_prompt=system_prompt,
30 query_wrapper_prompt=query_wrapper_prompt,
31 tokenizer_name="StabilityAI/stablelm-tuned-alpha-3b",
32 model_name="StabilityAI/stablelm-tuned-alpha-3b",
33 device_map="cuda",
34 stopping_ids=[50278, 50279, 50277, 1, 0],
35 tokenizer_kwargs={"max_length": 1028},
36 # uncomment this if using CUDA to reduce memory usage
37 model_kwargs={"torch_dtype": torch.float16}
38 )

File /opt/conda/envs/sagemaker-distribution/lib/python3.8/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.init()

ValidationError: 1 validation error for HuggingFaceLLM
query_wrapper_prompt
str type expected (type=type_error.str)

2 comments

kkapa.ai

Please stand by @Ftm as I search the knowledge base. I normally take about 20 seconds or so. :wave:

kkapa.ai

----------------------
The error message you're seeing is indicating that the query_wrapper_prompt parameter is expecting a string, but it's receiving an instance of SimpleInputPrompt.

In the examples provided in the extracts, the query_wrapper_prompt is initialized as an instance of SimpleInputPrompt, but when it's passed to the HuggingFaceLLM constructor, it's used directly as a string.

Here's an example from the extracts:

Plain Text

from llama_index.prompts.prompts import SimpleInputPrompt

query_wrapper_prompt = SimpleInputPrompt("<|USER|>{query_str}<|ASSISTANT|>")

And then it's used in the HuggingFaceLLM constructor:

Plain Text

llm = HuggingFaceLLM(
    ...
    query_wrapper_prompt=query_wrapper_prompt,
    ...
)

It seems like there might be a discrepancy between the way you're using query_wrapper_prompt and the way it's used in the examples.

You might want to try using the query_wrapper_prompt directly as a string when passing it to the HuggingFaceLLM constructor, like so:

Plain Text

query_wrapper_prompt = "<|USER|>{query_str}<|ASSISTANT|>"

llm = HuggingFaceLLM(
    ...
    query_wrapper_prompt=query_wrapper_prompt,
    ...
)

If this doesn't resolve your issue, I would recommend checking the LlamaIndex documentation () for more information on how to use the query_wrapper_prompt parameter.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

Add a reply

Find answers from the community

this is my query wrapper: query_wrapper_prompt = SimpleInputPrompt("<|USER|>{query_str}<|A