Anthropic default prompt in llamaindex
Hello gents.
I am running notebooks from the llamaindex -deeplearning short course on "Building and Evaluating advanced RAG"
I am running the sentence window retrieval notebook, where a sentence window retrieval technique , along with a reranker are used. Then the final context is passed to an llm for answer generation.
The notebook runs fine when the llm is left as GPT3.5.
The query is
What are the keys to building a career in AI?
GPT3.5's answer is:
Final Response: The keys to building a career in AI are learning foundational technical skills, working on projects, and finding a job, all of which is supported by being part of a community.
However, when i change the llm to Anthropic, claude-2, the llm is answering in a very odd way, and I suspect this is due to the default anthropic prompt in llamaindex.
Here is the response:
Final Response: Unfortunately I cannot directly reference the given context in my answer. However, it seems there are a few important elements discussed that could contribute to building a career in AI. Developing technical skills, working on projects, finding the right jobs, overcoming challenges like imposter syndrome, and being part of a supportive community all appear to play a role. Consistently making progress and effort each day also seems to be a key mindset. I apologize that I cannot directly reference the context, but I aimed to provide a helpful perspective based on the information provided.
I used get_prompts() to see what is the prompt being used:
Context information is below.
---------------------
{context_str}
---------------------
Given the context information and not prior knowledge, answer the query.
Query: {query_str}
Answer: However this prompt format doesn't work well with Claude, that's why i am suspecting its the prompt.
anyone else noticed this? should i just customize the prompt?
