----------------------
When developing your LLM application with LlamaIndex, you may encounter situations where your data is extremely specific (medical, legal, scientific, financial, or other documents with technical lingo) and falls out of the domain. Here are some strategies you can use to improve the results:
- Try out alternate embeddings: You can check the MTEB Leaderboard or configure a local embedding model. Here are the steps to configure a local embedding model: Local Embedding Models ()
- Test out fine-tuning of embeddings: Tools like setfit () can be used for this purpose. Anecdotally, retrieval accuracy has been seen to improve by ~12% by curating a small annotated dataset from production data. Even synthetic data generation without human labels has been shown to improve retrieval metrics across similar documents in train / val sets.
- Test out sparse retrieval methods: Methods like ColBERT, SPLADE have been shown to generalize well to out of domain data. Some of these methods are starting to be available in some enterprise systems like Elastic Search’s ELSeR ().
- Evaluate the changes: You can check out the evaluation principles guide () on how you might evaluate the above changes.
Remember, these are just strategies and may not completely restrict the response from the query if the input was out of the domain. However, they can significantly improve the retrieval accuracy.
