Anthropic

At a glance

The community members are discussing an issue where their working code is now throwing an AttributeError: 'Anthropic' object has no attribute 'get_tokenizer'. The issue seems to be related to an update in the Anthropic client library, which broke the integration. The community members provide several suggestions to resolve the issue, including:

- Using the latest version of the Anthropic and the integration libraries to fix the issue - Resolving dependency conflicts when upgrading the llama_index package - Setting the ANTHROPIC_API_KEY environment variable and using the correct Anthropic model before getting the tokenizer - Updating the documentation in the llama_index repository, as it seems to be outdated

There is no explicitly marked answer in the comments.

Useful resources

CCharles T

What-the-heck : working code now throwing: AttributeError: 'Anthropic' object has no attribute 'get_tokenizer'
Oh... hrmm... https://discord.com/channels/1059199217496772688/1059200010622873741/1305527139973730376

10 comments

LLogan M

The anthropic client lib updated and broke the integration. It's fixed if you use the latest version of anthropic and the integration

CCharles T

shoot "pip install -U llama_index" hit errors (sometimes I hate python pkg mgmt)

Plain Text

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
llama-index-llms-ollama 0.3.2 requires llama-index-core<0.12.0,>=0.11.0, but you have llama-index-core 0.12.13 which is incompatible.                                             llama-index-llms-anthropic 0.3.1 requires anthropic[vertex]<0.29.0,>=0.26.2, but you have anthropic 0.44.0 which is incompatible.
llama-index-llms-anthropic 0.3.1 requires llama-index-core<0.12.0,>=0.11.0, but you have llama-index-core 0.12.13 which is incompatible.

CCharles T

This worked
pip install -U llama_index.llms.anthropic
but other stuff is busted

Plain Text

llama_index.core.workflow.errors.WorkflowRuntimeError: Error in step 'prepare_chat_history': "Could not resolve authentication method. Expected either api_key or auth_token to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted"

CCharles T

so now the tokenizer needs the anthropic api key:

Plain Text

os.environ["ANTHROPIC_API_KEY"] = "sk-ant-api03-ntK

before the call to Anthropic().tokenizer

CCharles T

also the Anthropic() call defaults to claude-2.1, so you need your code to do the instantiation with a good model before getting the tokenizer, e.g.

Plain Text

    llm = Anthropic(model="claude-3-haiku-20240307", max_tokens=MAX_TOKENS)
    tokenizer = llm.tokenizer
    Settings.tokenizer = tokenizer

where the Settings is for llama_index.core ...

CCharles T

I think that makes sense, since I supposed the tokenizer could need to match the model (although it doesn't change often).
I'm not sure where I got the code originally .. hope the examples in the docs are okay.

CCharles T

sigh, okay nope.
https://github.com/run-llama/llama_index/blob/3cb06f88b181e2528cacb5f9db4d96dc905dfed5/llama-index-integrations/llms/llama-index-llms-anthropic/README.md?plain=1
this needs an update

LLogan M

Yea that needs a tweak it looks like. But besides that, seems all is well

LLogan M

And yea i know, confusing that anthropic would update the tokenizer like that 🤷‍♂️

For future reference too, every integration is a package. llama-index isn't even a real package, just a wrapper on some starter installs

CCharles T

this is probably more about my environment because the original example worked in a jupyter notebook.
I'm using workflows and async which may be causing some of the issue, but it's weird...
😥
https://github.com/run-llama/llama_index/pull/17607/files

Add a reply

Find answers from the community

Anthropic