Hello guys! Just upgraded the LLamaIndex

Hello guys! Just upgraded the LLamaIndex and all the libraries but still get this error when trying to use gpt-4o:

Plain Text

KeyError: 'Could not automatically map gpt-4o to a tokeniser. Please use `tiktok.get_encoding` to explicitly get the tokeniser you expect.'

The code:

Plain Text

        token_counter = TokenCountingHandler(
            tokenizer=tiktoken.encoding_for_model(model_name).encode,
            verbose=False  
        )

UPD After upgrading tiktoken the problem has gone!

Find answers from the community

Hello guys! Just upgraded the LLamaIndex