Find answers from the community

Updated 3 months ago

None of PyTorch, Tensorflow, Flax

I am seeing this error in production
Plain Text
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

Is this related to llama-index? I have no problem with everything locally
L
s
22 comments
Is this actually an error? I see it in some examples notebooks but it looks like a log or a warning

it doesn't appear in the code base at least
i dont know, but in production the request times out 100% of the time 😦
locally its 100%, but in prod (using railway)
Plain Text
one of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Token indices sequence length is longer than the specified maximum sequence length for this model (3335 > 1024). Running this sequence through the model will result in indexing errors
[2023-03-02 02:48:54 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:88)
[2023-03-02 02:48:54 +0000] [88] [INFO] Worker exiting (pid: 88)
[2023-03-02 02:48:54 +0000] [124] [INFO] Booting worker with pid: 124
I am also seeing this...so maybe its this
Token indices sequence length is longer than the specified maximum sequence length for this model (3335 > 1024). Running this sequence through the model will result in indexing errors
but i'm very confused, because this all works locally lol
the token length error seems more concerning πŸ‘€

Which OpenAI model are you using? I see the max length is 1024, are you using a prompt helper to customize the index/queries to match that length?
da-vinci-003, I have not customized the length limits at all 😬 , I haven't needed to locally
im confused how that only became an issue when hosted on another server though? Surely the API call is agnostic?
thanks for that resource! I want to dive into fine tuning this once i get the basic circuit published
i'm gonna try that out, but I'm not sure if thats the issue i'm facing
The only reason you can see it being a problem now is because it suddenly encountered a large document?
thats my best guess lol
haha its reading the same document hosted on S3
oof 😦 also extra strange, because I thought davinci-003 had a max limit of 4096 ...
hmmm maybe in the prod environment its defaulting to davinci-002?
If you are able to ssh or something onto prod, you can try running something like print(index._llm_predictor._llm.model_name)
very hacky, but just trying to think of how to narrow down the problem
thanks @Logan M , after some deep, brutal investigation, it looks like my python environment is different in my local vs production instance, and is pulling from a separate set of dependencies
absolutely brutal lol
Oof

Yea, I was only half joking about the requirements.txt

I've ran into this (and similar things) in prod before with python. Every dependency needs a version or a package will just randomly update and break everything lol
Glad you got it figured out though!
Add a reply
Sign up and join the conversation on Discord