Input size

Hi,

I have an simple vector index that I created with chunk_size_limit=1024

The input prompt itself that I pass to query function is ~5000 tokens. I tried using prompt_helper (below) to create chunks but I think that is useful when you create an index, not while making the query call.

        index = GPTListIndex.load_from_disk('./index.json')

        max_input_size = 4096

        num_output = 256

        max_chunk_overlap = 50

        prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)

        llm_predictor = LLMPredictor(llm=OpenAI(temperature=0, model_name="text-davinci-002"))

        service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)

        response = index.query(query_str, mode="default", response_mode="default",service_context=service_context)

The error that I'm getting is:
Got a larger chunk overlap (200) than chunk size (-2877), should be smaller.

Any suggestions?

Find answers from the community

Input size