The community member in the post is asking what happens if the content they want to embed exceeds the max token limit of the embedding model. The kapa bot suggests that the model will only consider the first max_length tokens and ignore the rest. The comments indicate that some models may truncate and only calculate embeddings for the first X tokens, while others may error out. However, there is no definitive answer provided.
Hi everyone. There is a max token limit for every embedding model. So if the size of the content I want to embed exceeds that token limit, what will happen? The kapa bot says that the model will only consider the first max_length tokens and ignore the rest. Is that the correct answer?
