Log in
Log into community
Find answers from the community
View all posts
Related posts
Did this answer your question?
π
π
π
Powered by
Hall
Inactive
Updated 10 months ago
0
Follow
was looking to finetune an adapter as
was looking to finetune an adapter as
Inactive
0
Follow
s
shawtyisaten
10 months ago
Β·
was looking to finetune an adapter as detailed here,
https://docs.llamaindex.ai/en/stable/examples/finetuning/embeddings/finetune_embedding_adapter.html
but it seems that hit rate is lower than if we were to use ada-2 model. doesn't higher hit rate mean that model is better at retrieving the right documents?
L
s
14 comments
Share
Open in Discord
L
Logan M
10 months ago
I think its show the ada is better than bge, but bge can be slightly better with fine-tuning (for this specific dataset)
L
Logan M
10 months ago
(tbh I just quickly skimmed it lol)
s
shawtyisaten
10 months ago
what metrics show that bge can be slightly better in this article? both hit rate and mrr is lower. (0.803797 vs 0.87088, 0.667426 vs 0.728840 at best)
s
shawtyisaten
10 months ago
don't understand the point of this article if it yields poorer metrics
s
shawtyisaten
10 months ago
or am i misunderstanding something?
L
Logan M
10 months ago
bge
has hit rate of 0.78, then with the fine-tuned adaptor its 0.79
L
Logan M
10 months ago
Attachment
s
shawtyisaten
10 months ago
right
L
Logan M
10 months ago
so small increase when finetuning an adaptor for bge
s
shawtyisaten
10 months ago
but i was under the impression that it would be better than ada
s
shawtyisaten
10 months ago
is it supposed to be better if we fine-tune with some other text embedding models?
L
Logan M
10 months ago
What gave you that impression? Especially since it's bge-small, its quite impressive performance compared to ada imo
People want local embeddings, and they want them to be good. This is one option to improve performance of local embeddings π€
s
shawtyisaten
10 months ago
i see. i was under the impression that this was supposed to tell us how to fine-tune an embedding model that's better than ada.
s
shawtyisaten
10 months ago
thank you for your answer.
Add a reply
Sign up and join the conversation on Discord
Join on Discord