Hi everyone, I am planning to use

Under the hood, it's using the fit() method provided by sentence transformers

Not 100% sure, but from my understanding this fine-tunes all model weights. Embedding models are usually small, so no need for LoRa

ok thanks !

I will look at Sentence Transformer documentation then

the SentenceTransformersFinetuneEngine class in the codebase is used for fine-tuning Sentence Transformers models. It's not a specific type of training like LoRa (Long Range) training. Instead, it's a classic training approach where all the weights of the model are adjusted based on the training data.

The finetune method in this class calls the fit function of the Sentence Transformers model with several parameters including the training data, the number of epochs, the warmup steps, the output path for the model, whether to show a progress bar, the evaluator, and the evaluation steps.

This method doesn't specify which layers of the model are being fine-tuned, so it's safe to assume that all layers of the model are being fine-tuned. This is typical in transfer learning scenarios where a pre-trained model is fine-tuned on a specific task.

Note that the actual implementation of how the fine-tuning is performed would be in the classes that inherit from BaseEmbeddingFinetuneEngine, such as the SentenceTransformersFinetuneEngine class. The BaseEmbeddingFinetuneEngine class itself is an abstract base class and does not provide an implementation for the finetune method.

this is my best guess
cc: @Logan M can keep me honest