Log in
Log into community
Find answers from the community
View all posts
Related posts
Was this helpful?
π
π
π
Powered by
Hall
Inactive
Updated last year
0
Follow
was looking to finetune an adapter as
was looking to finetune an adapter as
Inactive
0
Follow
At a glance
s
shawtyisaten
last year
Β·
was looking to finetune an adapter as detailed here,
https://docs.llamaindex.ai/en/stable/examples/finetuning/embeddings/finetune_embedding_adapter.html
but it seems that hit rate is lower than if we were to use ada-2 model. doesn't higher hit rate mean that model is better at retrieving the right documents?
L
s
14 comments
Share
Open in Discord
L
Logan M
last year
I think its show the ada is better than bge, but bge can be slightly better with fine-tuning (for this specific dataset)
L
Logan M
last year
(tbh I just quickly skimmed it lol)
s
shawtyisaten
last year
what metrics show that bge can be slightly better in this article? both hit rate and mrr is lower. (0.803797 vs 0.87088, 0.667426 vs 0.728840 at best)
s
shawtyisaten
last year
don't understand the point of this article if it yields poorer metrics
s
shawtyisaten
last year
or am i misunderstanding something?
L
Logan M
last year
bge
has hit rate of 0.78, then with the fine-tuned adaptor its 0.79
L
Logan M
last year
Attachment
s
shawtyisaten
last year
right
L
Logan M
last year
so small increase when finetuning an adaptor for bge
s
shawtyisaten
last year
but i was under the impression that it would be better than ada
s
shawtyisaten
last year
is it supposed to be better if we fine-tune with some other text embedding models?
L
Logan M
last year
What gave you that impression? Especially since it's bge-small, its quite impressive performance compared to ada imo
People want local embeddings, and they want them to be good. This is one option to improve performance of local embeddings π€
s
shawtyisaten
last year
i see. i was under the impression that this was supposed to tell us how to fine-tune an embedding model that's better than ada.
s
shawtyisaten
last year
thank you for your answer.
Add a reply
Sign up and join the conversation on Discord
Join on Discord