Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
π
π
π
Powered by
Hall
Inactive
Updated 3 months ago
0
Follow
Hi friends How to improve response time
Hi friends How to improve response time
Inactive
0
Follow
o
openmind
2 years ago
Β·
Hi, friends, How to improve response time from query in llama-index?
L
o
3 comments
Share
Open in Discord
L
Logan M
2 years ago
Either setting a smaller chunk_size in the service_context, avoiding using complex index structures if possible, or enabling streaming, will improve the speed (or at least make it feel faster, i.e. with streaming)
o
openmind
2 years ago
for chunk_size, what would be ideal?
L
Logan M
2 years ago
usually the default (1024) is the best balance between speed and quality of generated embeddings. I wouldn't go much lower than 512
Add a reply
Sign up and join the conversation on Discord
Join on Discord