Hi all. Can we currently use the gpt-4o-

At a glance

The post asks if the GPT-4o-mini model can currently be used. The comments suggest that the model is supported, and provide steps to upgrade the necessary package. However, a community member encountered an error with the token counter, which they were able to resolve by uninstalling and reinstalling the llama-index package.

Another community member raised a concern about the CondensePlusContextChatEngine modifying the wording of the provided content, unlike the RetrieverQueryEngine. The response suggests using skip_condense=True when instantiating the chat engine to avoid this issue. However, the community member still experienced problems with the returned links being incorrect.

BBrent

Hi all. Can we currently use the gpt-4o-mini model?

12 comments

WWhiteFang_Jr

Yes its supported since day 0.
You just need to upgrade the openai llm package:
pip install -U llama-index-llms-openai

BBrent

Thanks. It worked after your command. But there is a problem: the token counter part shows the error AttributeError: 'ChatCompletion' object has no attribute 'get'

ttomoe

May I see what is your chat completion code?

WWhiteFang_Jr

Are you using Chatcompletion directly? yea hacing more detail on the error or code would be helpful

BBrent

I solved it by:

pip uninstall llama-index # remove any possible global install
python -m venv venv
source venv/bin/activate
pip install llama-index

BBrent

@WhiteFang_Jr Can I ask you how the CondensePlusContextChatEngine can only accurately respond to the provided content? I am facing a situation where the RetrieverQueryEngine correctly answers all the content I provide, but the CondensePlusContextChatEngine responds with the same content but with modified wording. How can I limit this since it's not always possible to allow modifications?

WWhiteFang_Jr

CondensePlusContext converts user query based on chat history. I think if you want to pass only your query and dont want to modify it , pass skip_condense=True while instantiating the chat engine

BBrent

I have tried and found it to be the same. Specifically, my data returns a link, but with the CondensePlusContext, it returns the wrong link, so the correct link cannot be opened.

WWhiteFang_Jr

Is the wording changing again?

Can you check if the nodes retrieved are same now as that of RetrieverQueryEngine

BBrent

I use the same index and do not modify the nodes inside.

WWhiteFang_Jr

Yea no, I mean your problem statement was that when you used RetrieverQueryEngine the nodes were diff and when you are doing the same with a chat engine the nodes are coming different as it was changing the wording of your query

is that right?

BBrent

You have a point. I only use one index and there are two ways to use it: RetrieverQueryEngine or CondensePlusContextChatEngine.

Add a reply

Find answers from the community

Hi all. Can we currently use the gpt-4o-