Find answers from the community

S
SeaCat
Offline, last seen 3 months ago
Joined September 25, 2024
Hello guys! Just upgraded the LLamaIndex and all the libraries but still get this error when trying to use gpt-4o:
Plain Text
KeyError: 'Could not automatically map gpt-4o to a tokeniser. Please use `tiktok.get_encoding` to explicitly get the tokeniser you expect.'

The code:
Plain Text
        token_counter = TokenCountingHandler(
            tokenizer=tiktoken.encoding_for_model(model_name).encode,
            verbose=False  
        )

UPD After upgrading tiktoken the problem has gone!
1 comment
L