None of PyTorch, Tensorflow, Flax

I am seeing this error in production

Plain Text

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

Is this related to llama-index? I have no problem with everything locally

22 comments

LLogan M

Is this actually an error? I see it in some examples notebooks but it looks like a log or a warning

it doesn't appear in the code base at least

ssmokeoX

i dont know, but in production the request times out 100% of the time 😦

ssmokeoX

locally its 100%, but in prod (using railway)

Plain Text

one of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Token indices sequence length is longer than the specified maximum sequence length for this model (3335 > 1024). Running this sequence through the model will result in indexing errors
[2023-03-02 02:48:54 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:88)
[2023-03-02 02:48:54 +0000] [88] [INFO] Worker exiting (pid: 88)
[2023-03-02 02:48:54 +0000] [124] [INFO] Booting worker with pid: 124

ssmokeoX

I am also seeing this...so maybe its this

Token indices sequence length is longer than the specified maximum sequence length for this model (3335 > 1024). Running this sequence through the model will result in indexing errors

ssmokeoX

but i'm very confused, because this all works locally lol

LLogan M

the token length error seems more concerning 👀

Which OpenAI model are you using? I see the max length is 1024, are you using a prompt helper to customize the index/queries to match that length?

LLogan M

See this example: https://gpt-index.readthedocs.io/en/latest/how_to/custom_llms.html#example-fine-grained-control-over-all-parameters

ssmokeoX

da-vinci-003, I have not customized the length limits at all 😬 , I haven't needed to locally

ssmokeoX

im confused how that only became an issue when hosted on another server though? Surely the API call is agnostic?

ssmokeoX

thanks for that resource! I want to dive into fine tuning this once i get the basic circuit published

ssmokeoX

i'm gonna try that out, but I'm not sure if thats the issue i'm facing

LLogan M

The only reason you can see it being a problem now is because it suddenly encountered a large document?

LLogan M

thats my best guess lol

ssmokeoX

haha its reading the same document hosted on S3

LLogan M

oof 😦 also extra strange, because I thought davinci-003 had a max limit of 4096 ...

ssmokeoX

hmmm maybe in the prod environment its defaulting to davinci-002?

LLogan M

If you are able to ssh or something onto prod, you can try running something like print(index._llm_predictor._llm.model_name)

LLogan M

very hacky, but just trying to think of how to narrow down the problem

ssmokeoX

thanks @Logan M , after some deep, brutal investigation, it looks like my python environment is different in my local vs production instance, and is pulling from a separate set of dependencies

ssmokeoX

absolutely brutal lol

LLogan M

Oof

Yea, I was only half joking about the requirements.txt

I've ran into this (and similar things) in prod before with python. Every dependency needs a version or a package will just randomly update and break everything lol

LLogan M

Glad you got it figured out though!

Add a reply

Find answers from the community

None of PyTorch, Tensorflow, Flax