Do you have a more complete example? I could run a test or two on my end and see if I can figure it out, if I know the index and query commands
Here's technically what I'm trying to do... I have this working in langchain completely but trying to migrate over to gpt index
Maybe let's just start with this summary query
prompt = """Write three concise and unique summaries of the following.\n Return the format in a JSON Object {{"summaries": ["Summary 1", "Summary 2", "Summary 3"]}}:"""
refine_template = (
"{query_str}"
"We have provided existing summaries in the same json format: {existing_answer}\n"
"We have the opportunity to refine the existing summary"
"------------\n"
"{context_msg}\n"
"------------\n"
"If there is no better summary please just return the summary in JSON Object mapping like this: {{\"summaries\": [\"Summary 1\", \"Summary 2\", \"Summary 3\"]}}\n"
"Given the new context, please write three unqiue summaries in JSON format with a maximum of 160 words that include all relevant topics and subjects."
"Try to include who is speaking.\n"
"Return the format in valid JSON Object such as {{\"summaries\": [\"Summary 1\", \"Summary 2\", \"Summary 3\"]}}\n"
"return only valid JSON: "
)
refine_prompt = RefinePrompt(
template=refine_template,
)
return self.index.query(prompt, refine_template=refine_prompt, response_mode="tree_summarize")this is the index code
text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
chunk_size=1500, chunk_overlap=600)
text_list_character = text_splitter.split_text(self.transcription)
documents = [Document(t)
for t in text_list_character]
llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.1, model_name="text-davinci-003"))
# define prompt helper
# set maximum input size
max_input_size = 8000
num_output = 500
max_chunk_overlap=50
prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)
index = GPTSimpleVectorIndex(
documents, llm_predictor=llm_predictor, prompt_helper=prompt_helper
)@Logan M thank you
These details will help a lot! I'll ping back in a few hours and let you know how it goes lol
@jerryjliu0 any idea my friend 🙂 ?
@bmax Just finished running a few tests. I think you are slightly misusing the vector index.
The vector index returns the most similar chunks to the prompt. But in your code, the prompt is an instruction, so who knows what the vector index will do 😅
You might have better luck with the attached example (I'm using the Paul Graham essay data included in the repo). Here, I use a QA template based off your original prompt, as well as the same refine template. When I queried about Interleaf (a company mentioned in the essay), the results looked not bad. I formated the response in the screenshot to make it a little easier to read
hmm the vector index has actually been working decent!
If you want to skip the vector matching, simply replacing the vector index with a list index in your original code seems to also work. Rather than querying a specific topic, it ends up summarizing the entire index (... I think haha)
I want to do mostly generative AI off of the large transcript.
So I want to generate long form content, twitter threads, summaries, titles.
BUT I also want to do something like a "What did guest X say about Topic Y?"
Perhaps you need to use two different indexes then? Like a vector and list index? Just spitballing 😅
@Logan M you queried interleaf, but, I'm trying to summarize the whole text, not just a specific part.
@bmax right. And if your are trying to summarize the whole thing, I would use a list index instead. It should drop right into the original code you sent
I'll try your code and everything and my life haha thank. you
@Logan M GPTListIndex gives me same error
Code that I originally sent but changed index to
index = GPTListIndex(
documents, llm_predictor=llm_predictor, prompt_helper=prompt_helper
)thanks for the taking the time
I don't mind having two indexes, and the QAPrompt I think will work for a lot of my other use cases but right now I'm just trying to generate summaries of the whole index
Here's a working minimum example with the list index and your prompts/text splitter. And here's the output it printed at the end of the code. (I had to lower the output size to 4096 to not go over openAI's limit)
There must be a difference somewhere compared to the code you have, I didn't run into any errors running that 🤔
ahh wow... I got a step closer...
You're actually not using refine_prompt anywhere! so if I just pass in refine_template... it works closer, i think, I now get this error:
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens, however you requested 6863 tokens (6607 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.
weird though because text-davinci-3 is 8000 tokens I though.
@jerryjliu0 ^ friend if you're around ❤️
same exact code!! LOL WTF
ah ha, okay, thank you @jerryjliu0 -- GPTList isn't giving as good as summaries as GPTVector was, is there a way to do a vector index w/ a refine template and summarize the whole document?
I promise to repay all of the help i've gotten haha
@bmax gpt list index isn't giving you as good summaries as gpt vector index was? that's surprising, since list index goes through the entire set of nodes instead of just fetching top k. have you tried response_mode="tree_summarize" during query-time? that might help give you better results
hmm it is, but, i'm not sure it's using my refine_template.
if you see in code above, we're passing in refine_template (which is just a string) directly, not wrapping it in RefinePrompt
ahhhh i see. so the response_mode="tree_summarize" only uses the QuestionAnswer prompt template, not the refine template. I realize this is super confusing and not in the docs
ahh and there's no way to use the QuestionAnswer Prompt as the initial prompt?
i.e pass it in as first param
you can, just pass in text_qa_template=prompt_template during the index.query call
response= self.index.query(text_qa_template=prompt, refine_template=refine_template, response_mode="tree_summarize") ?
yep, the refine_template won't be used so you don't need to pass that in
langchainapi-langchain-1 | NameError: name 'prompt_template' is not defined
excuse my docker image name in the logs:
langchainapi-langchain-1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2548, in __call__
langchainapi-langchain-1 | return self.wsgi_app(environ, start_response)
langchainapi-langchain-1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
langchainapi-langchain-1 | response = self.handle_exception(e)
langchainapi-langchain-1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2525, in wsgi_app
langchainapi-langchain-1 | response = self.full_dispatch_request()
langchainapi-langchain-1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1822, in full_dispatch_request
langchainapi-langchain-1 | rv = self.handle_user_exception(e)
langchainapi-langchain-1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1820, in full_dispatch_request
langchainapi-langchain-1 | rv = self.dispatch_request()
langchainapi-langchain-1 | File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1796, in dispatch_request
langchainapi-langchain-1 | return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
langchainapi-langchain-1 | File "/src/main.py", line 108, in embed
langchainapi-langchain-1 | return GPT(data).api()
langchainapi-langchain-1 | File "/src/gpt.py", line 77, in api
langchainapi-langchain-1 | results = self.handleType()
langchainapi-langchain-1 | File "/src/gpt.py", line 73, in handleType
langchainapi-langchain-1 | summaries = self.summary()
langchainapi-langchain-1 | File "/src/gpt.py", line 57, in summary
langchainapi-langchain-1 | response= self.index.query(text_qa_template=prompt, refine_template=refine_template, response_mode="tree_summarize")
langchainapi-langchain-1 | NameError: name 'prompt_template' is not defined
