Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
😞
😐
😃
Powered by
Hall
Inactive
Updated 2 months ago
0
Follow
Nodes
Nodes
Inactive
0
Follow
i
isaackogan
9 months ago
·
i think the answer is no but just checking
L
i
5 comments
Share
Open in Discord
L
Logan M
9 months ago
We really should add that lol
Just that all the current ones are CPU bound for the most part lol
i
isaackogan
9 months ago
@Logan M I can take a look today to see what it would require
i
isaackogan
9 months ago
From what I’m seeing they’re not all cpu bound tho
i
isaackogan
9 months ago
It’s a lot of I/O with LLMs or re-ranker models over HTTP
L
Logan M
9 months ago
Yes very true. I think it's as simple as adding async to the base class, and ensuring async gets used when needed
Add a reply
Sign up and join the conversation on Discord
Join on Discord