Not sure I understood your question I m

At a glance

Not sure I understood your question. I'm asking some questions that can be found in that specific document. What I found right now: if I use the gpt-3.5-turbo model, it's not working but works with some other, see the screenshot:

Attachment

15 comments

LLogan M

Was just wondering if you were asking questions that were more about location (i.e. stuff like "What does Rule 12 paragraph 2 say?" is too specific). But your queries make sense, it seems fine.

GPT-3.5 can be improved somewhat if you use a new refine prompt template. I've been experimenting with one lately. OpenAI "upgraded" gpt-3.5 recently and it struggles with the refine process in llama index.

One, sec, let me get the code to change the refine prompt I had

LLogan M

Plain Text

from langchain.prompts.chat import (
    AIMessagePromptTemplate,
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
)

from llama_index.prompts.prompts import RefinePrompt

# Refine Prompt
CHAT_REFINE_PROMPT_TMPL_MSGS = [
    HumanMessagePromptTemplate.from_template("{query_str}"),
    AIMessagePromptTemplate.from_template("{existing_answer}"),
    HumanMessagePromptTemplate.from_template(
        "I have more context below which can be used "
        "(only if needed) to update your previous answer.\n"
        "------------\n"
        "{context_msg}\n"
        "------------\n"
        "Given the new context, update the previous answer to better "
        "answer my previous query."
        "If the previous answer remains the same, repeat it verbatim. "
        "Never reference the new context or my previous query directly.",
    ),
]


CHAT_REFINE_PROMPT_LC = ChatPromptTemplate.from_messages(CHAT_REFINE_PROMPT_TMPL_MSGS)
CHAT_REFINE_PROMPT = RefinePrompt.from_langchain_prompt(CHAT_REFINE_PROMPT_LC)
...
index.query(..., refine_template=CHAT_REFINE_PROMPT)

SSeaCat

Oh, this sounds interesting! I should try and see if it helps, thanks!!

SSeaCat

One more question, is the number of used tokens (~4,000) okay for this one simple question? What does it depend on?

LLogan M

That sounds about right

It depends on the chunk_size_limit and the similarity top k

Actually as I say this, you might want to customize those lol

service_context = service_context.from_defaults(..., chunk_size_limit=1024)

index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)

index.query(..., similarity_top_k=3, response_mode="compact")

LLogan M

Those settings may help (in addition to that template!)

LLogan M

In the newer versions, the defaults are like that (but newer versions also changed a lot of interfaces lol)

SSeaCat

Well, nice, I have to try to change the defaults. When you mention "new versions" do you mean new versions of what exactly?

LLogan M

new versions of llama index (I noticed you are on an older version based on your code)

SSeaCat

Ahhh I will check the version, thanks for this advice!

SSeaCat

Also, you know what. I actually found the reason for the bad results. It turned out that the PDF reader (I copied the code from LlamaHub) worked incorrectly, it just skipped big chunks of text, and read other text with pretty low quality. I will do more investigation to see what is the reason. Thanks for your help!

LLogan M

Just be warned if you upgrade, many things changed 😅 Definitely re-read the docs for a refresh if you do upgrade

LLogan M

That makes sense!

SSeaCat

😬

SSeaCat

Yeah, absolutely, will read the docs!

Add a reply

Find answers from the community

Not sure I understood your question I m