Response Synthesizer
in the official docs. Is it the right place to learn about summarization strategies in LlamaIndex?from llama_index.core.response_synthesizers import TreeSummarize summarizer = TreeSummarize(llm=llm, use_async=True) response = summarizer.get_response("Summarize the text", ["text1", "text2", ...])
TreeSummarize
class to summarize a list of my docs, I'm getting the following error:ValidationError: 1 validation error for SummaryAndThemes __root__ Unterminated string starting at: line 1 column 618 (char 617) (type=value_error.jsondecode; msg=Unterminated string starting at; doc={"summary":"The comments reflect concerns about the rising cost of living, uncertainties about the future, challenges in coping with inflation, and the need for more support from the government. Many individuals express a lack of confidence in the current financial assistance provided and worry about job security and healthcare costs. There is a general sentiment of appreciation for the government's efforts but a desire for more substantial and long-term solutions to address the ongoing economic challenges.","themes":["Rising cost of living","Uncertainty about the future","Challenges in coping with inflation","Need; pos=617; lineno=1; colno=618)
summarizer.get_response
?class SummaryAndThemes(BaseModel): """Data model for a summary and themes.""" summary: str themes: List[str] summarizer = TreeSummarize(verbose=True, output_cls=SummaryAndThemes) response = summarizer.get_response("""These are user comments, separated by new lines. \ Summarize them and list down key themes mentioned in these comments. """, comment_chunks # each chunk contain ~4096 tokens (I'm using gpt-4o) )
llm = OpenAI(..., max_tokens=1024)
llm = OpenAI(model="gpt-3.5-turbo", max_tokens=1024) summarizer = TreeSummarize(llm=llm, output_cls=SummaryAndThemes)
llm
argument to TreeSummarize
. Even though I set Settings.llm=OpenAI(model="gpt-4o")
beforehand, I am having this error:BadRequestError: Error code: 400 - {'error': {'message': "This model's maximum context length is 16385 tokens. However, you requested 16543 tokens (15446 in the messages, 73 in the functions, and 1024 in the completion). Please reduce the length of the messages, functions, or completion.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}
gpt-3.5
. That's weird though, because yesterday I was getting a different error.summarizer = TreeSummarize(verbose=True, output_cls=SummaryAndThemes, llm=Settings.llm)
summarizer.get_response
function 10 times for testing, and not getting the error.