Hello I m new to LlamaIndex and I m

xxrt

Hello, I'm new to LlamaIndex and I'm trying to use prompt template. what I'm doing wrong? because I get only bad answers 😦

Plain Text

template = (
    "We have provided context information below. \n"
    "---------------------\n"
    "{context_str}"
    "\n---------------------\n"
    "Given this information, please answer the question: {query_str}\n"
)
custom_qa_prompt = PromptTemplate(template)

query_engine = index.as_query_engine(
    service_context=service_context,
    similarity_top_k=similarity_top_k,
    text_qa_template=custom_qa_prompt,
    verbose=True
)

26 comments

LLogan M

I think you only need single brackets on the template no?

xxrt

Super, thank you, I think is working now 🙂 This template will work the same for any LLM models ?

LLogan M

Depends -- some LLMs, especially open-source LLMs, need a very specific input/prompt format

I would expect the above to work well with the best LLMs though -- openai, anthropic, palm

xxrt

There is any difference if I use like this in retriver ?

Plain Text

     retriever = index.as_retriever(
        retriever_mode=embed_model,
        similarity_top_k=similarity_top_k,
        text_qa_template=custom_qa_prompt, )
    query_engine = RetrieverQueryEngine(
        retriever,
    )

LLogan M

the prompt is not used in a retriever -- should pass that into the query engine

Plain Text

retriever = index.as_retriever(
    retriever_mode=embed_model,  # not 100% sure if this is valid, what is `embed_model` here?
    similarity_top_k=similarity_top_k,
)
query_engine = RetrieverQueryEngine.from_args(
    retriever,
    text_qa_template=custom_qa_prompt
)

xxrt

embed_model = LangchainEmbedding(
BedrockEmbeddings(
model_id="amazon.titan-e1t-medium",
)
)

xxrt

Why the answer is intrerrupted and is not completed ?

Plain Text

question: What is the document title?
llm_response:
Human: The document title is not explicitly stated in the provided context. Based on the context, this appears to be a draft framework service contract for outsourcing between the customer and a contractor. The title can be inferred from the

LLogan M

hmm nod idea lol

How did you setup the LLM? or did you not change the LLM settings?

xxrt

llm = Bedrock(model_id="anthropic.claude-v2")

xxrt

I know there should be a configuration where I can specify how many tokens response can have ? probably now there is a default ?

LLogan M

most likely. The LLM code itself doesn't really have a specific spot to specify the number of output tokens, maybe model_kwargs? https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/bedrock.py

xxrt

there will be support in LlamaIndex in the feature to AWS Bedrock models?

LLogan M

We support the langchain LLM. Maybe someday someone will add the intergration to llama-index natively

ttejesh bhalla

this works fine with langchain abstraction but how do i use it with a react agent

ttejesh bhalla

seems not to work with it

ttejesh bhalla

any ideas ?

ttejesh bhalla

it sends back some error about prompt not being in format of action and thought

LLogan M

if you pass the LLM directly to the agent, it needs the langchain wrapper

Plain Text

from llama_index.llms import LangChainLLM

llm = LangChainLLM(<bedrock_llm>)
agent = ReActAgent.from_tools(..., llm=llm)

ttejesh bhalla

i did this it still returned an error

ttejesh bhalla

the error was something like prompt needs to be in format of action and thought

LLogan M

Do you have the full error? Maybe the LLM just didn't follow the formatting instructions?

ttejesh bhalla

hey it seems to be an issue with the LangChainLLM wrapper

ttejesh bhalla

if i call it with langchain i even tried with azure open ai model from langchain and did the same wrapping if azure follows format in llamaindex why does it fail with a langchain wrapper

LLogan M

You've barely given me any details. Can you give some code snippets? A full traceback? I have no idea 🙂

ttejesh bhalla

Response:
Response:
Response:
Response:
Response:
Response:
Response:
Response:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-313-f7bfd2f34fdf> in <cell line: 1>()
----> 1 response = await agent2.astream_chat(
2 "hi how are you?"
3 )
4
5 response_gen = response.response_gen

3 frames
/usr/local/lib/python3.10/dist-packages/llama_index/agent/react/base.py in _extract_reasoning_step(self, output)
118 """
119 if output.message.content is None:
--> 120 raise ValueError("Got empty message.")
121 message_content = output.message.content
122 current_reasoning = []

ValueError: Got empty message.

LLogan M

what version of llama-index do you have? Does it work if you just use chat() ?

Add a reply

Find answers from the community

Hello I m new to LlamaIndex and I m