Hi everyone I m new to llamaindex and I

ssgaseretto

Hi everyone? I'm new to llamaindex and I was wondering if there are any examples for creating a simple Semantic Search engine, without any generation. Basically the idea would be to use something like Qdrant or any vectorstore, embed a bunch of data using something like e5 first. Then query some candidates with that same model and then reranking the candidates using something like a cross-encoder to rerank the candidates. Only that, it does not need to generate anything

2 comments

LLogan M

There's a few things to unpack there

Configuring an embedding model (defualt is openai text-ada-002)
https://gpt-index.readthedocs.io/en/latest/core_modules/model_modules/embeddings/usage_pattern.html#embedding-model-integrations

Configuring a vector store (i.e. qdrant, or you can use our default simple in-memory thing)
https://gpt-index.readthedocs.io/en/stable/examples/vector_stores/QdrantIndexDemo.html

Getting the retriever
retriever = index.as_retriever(similarity_top_k=2)

nodes = retriever.retrieve("query")

Post processing with re-ranking
https://gpt-index.readthedocs.io/en/stable/core_modules/query_modules/node_postprocessors/modules.html#sentencetransformerrerank

ssgaseretto

Thanks a lot, I'll look at this! The idea is to use sentence-transformers as much as posible, I see that llamaindex supports custom models for generation and embedding, so this is great!

Add a reply

Find answers from the community

Hi everyone I m new to llamaindex and I