Custom LLM

mmeeffe

Anyone could tell me why by using this exact code with gpt-4 as a model specified, I stil got usage only for gpt 3.5 turbo? How to use gpt-4 here?

https://github.com/aamyren/CalHacks2023/blob/133a8c144cfb925d400c125e37b60b5446d91e41/app.py#L3

I specified:

Plain Text

llm_predictor = LLMPredictor(
        llm=openai(temperature=0.7, model_name="gpt-4", max_tokens=num_outputs)
    )

but still it treat and use gpt 3.5 turbo from openai usage page

3 comments

EEmanuel Ferreira

https://docs.llamaindex.ai/en/stable/module_guides/models/llms/usage_custom.html#example-changing-the-underlying-llm

Try this way, LLMPredictor is a bit deprecated

mmeeffe

@Emanuel Ferreira this way?

Plain Text

from llama_index import (
    SimpleDirectoryReader,
    GPTVectorStoreIndex,
    ServiceContext,
    PromptHelper,
)
from langchain.chat_models import ChatOpenAI as openai

def construct_index(directory_path):
    max_input_size = 4096
    num_outputs = 512  # było 500
    max_chunk_overlap = 1
    chunk_size_limit = 600
    prompt_helper = PromptHelper(
        max_input_size,
        num_outputs,
        max_chunk_overlap,
        chunk_size_limit=chunk_size_limit,
    )

    # define LLM
    llm = openai(temperature=0.1, model="gpt-4")
    service_context = ServiceContext.from_defaults(llm=llm)
    # print model_name:
    documents = SimpleDirectoryReader(directory_path, num_files_limit=1).load_data()
    index = GPTVectorStoreIndex.from_documents(
        documents, service_context=service_context, prompt_helper=prompt_helper
    )
    index.storage_context.persist("index.json")

    return index

edit:

ok, it does use GPT4 - thanks for that!

WWhiteFang_Jr

Llamaindex gives llm support as well.

Plain Text

from llama_index.llms import OpenAI

llm = OpenAI(temperature=0.1, model="gpt-4")

Add a reply

Find answers from the community

Custom LLM