Is there a way to increase response

At a glance

The community member asked if there is a way to increase the response length with GPTSimpleVectorIndex, as they believe the standard max is 256 tokens. In the comments, another community member provided a code snippet that sets the maximum input size to 8000, the number of output tokens to 500, and the maximum chunk overlap to 50, using the PromptHelper class. Another community member suggested that there is an easier way to change the number of output tokens, depending on the language model being used, and provided a link to the relevant documentation.

Useful resources

TTeemu

Is there a way to increase response length with GPTSimpleVectorIndex? I think the standard max is 256 tokens?

4 comments

bbmax

Plain Text

      # define prompt helper
      # set maximum input size
      max_input_size = 8000
      num_output = 500
      max_chunk_overlap=50
      prompt_helper = PromptHelper(max_input_size, num_output, max_chunk_overlap)

bbmax

Plain Text

      return GPTSimpleVectorIndex(
        documents, llm_predictor=llm_predictor, prompt_helper=prompt_helper
      )

TTeemu

Thank you!

jjerryjliu0

yep! depending on your LLM there's an easier way to do that https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-changing-the-number-of-output-tokens-for-openai-cohere-ai21

Add a reply

Find answers from the community

Is there a way to increase response