Find answers from the community

Home
Members
MJThomas
M
MJThomas
Offline, last seen 3 months ago
Joined September 25, 2024
Hi, I am attempting to leverage a HuggingFaceEndpoint LLM, but am running into issue with the token count. I tried to set the paramater chunk_size in the service context, but it appears that this is not reducing the amount of tokens.I was wondering if there was anyway to reduce the token/ manage the amount of tokens used in a call? Any help would be appreciated.
6 comments
L
M
D