I'm in the process of updating my repo

TTorsten

I'm in the process of updating my repo to a newer version. It seems like the LLMPredictor class has been depreciated. I used this to set up my ServiceContext. What should I use for that now? I cannot find it in the docs.

18 comments

LLogan M

LLMPredictor was a do-nothing wrapper

LLogan M

you can setup like this

Plain Text

llm = OpenAI()
embed_model = OpenAIEmbedding()

index = VectorStoreIndex.from_documents(..., embed_model=embed_model)

index.as_query_engine(llm=llm)

Or you can set a global default

Plain Text

from llama_index.core import Settings

Settings.llm = llm
Settings.embed_model = embed_model

TTorsten

Thanks! Do you also know how I can combine this with the prompthelper?

LLogan M

prompt helper is old news, its not needed anymore. All its settings come from the LLM being used

LLogan M

If you really want, you can set Settings.prompt_helper=prompt_helper, but I kind of advocate against using it

TTorsten

Hmmm okay so if I want to limit the num_outputs for example I should do that during the embedding? Code for embedding currently looks like this. How to rewrite it?

LLogan M

You are on v0.10.x right? I would rewrite that as

Plain Text

llm = OpenAI(temperature=0.0, model_name="gpt-4-0613", max_tokens=3000)

documents = SimpleDirectoryReader(
  directory_path, 
  file_metadata=filename_fn
).load_data()

index = VectorStoreIndex.from_documents(
  documents
)

query_engine = index.as_query_engine(llm=llm)

Or if you don't want to worry about passing in the LLM

Plain Text

from llama_index.core import Settings

Settings.llm = llm

...
query_engine = index.as_query_engine()

TTorsten

Yes the old one was running on 0.8.50, now rewriting for 10.17

TTorsten

Thanks for the help! The upper snippet should work. I am curious what you import the sentence splitter for?

TTorsten

So how I'm doing it is I scrape and create the index using a jupyter notebook and then I load the acquired index in a repo where my app runs.

TTorsten

Also, gpt-4-0613 is okay for embedding, or do you recommend a different model?

LLogan M

ah ignore the splitter, I thought I would need it to rewrite but turns out I didn't lol

LLogan M

the LLM is not used for embedding, and embed_model is. It defaults to openai text-embedding-ada-002

TTorsten

Okay thanks, one last question. from which document length on should I change the default embedding parameters?

LLogan M

Sorry, what do you want to change? 👀

TTorsten

You said that the prompthelper is not needed anymore and that all settings come from the underlying llm. However the underlying llm doesn't influence the chunk_overlap_ratio for example. I assume you left it out because it's not necessary for most use cases. But how can I still pass this to the Vectorstore and when would you suggest I do this?

LLogan M

You could technically set this in the settings Settings.prompt_helper = ... -- but in 99.999% of use cases you don't need to change this lol

TTorsten

Okay I'll take your word for it haha. Thanks for all the help 🙂

Add a reply

Find answers from the community

I'm in the process of updating my repo