When using openai gpt4 turbo for llm in

권권씨 😮💨

When using openai gpt4 turbo for llm in llama index, if an embedding model is not specified, what embedding model is used by default for embedding?

Plain Text

chunk_sizes = [128, 256, 512]

nodes_list = []

vector_indices = []

for chunk_size in chunk_sizes:
    print(f"Chunk Size: {chunk_size}")
    splitter = SentenceSplitter(chunk_size=chunk_size, chunk_overlap=chunk_size // 2)

    nodes = splitter.get_nodes_from_documents(docs)

    for node in nodes:
        node.metadata["chunk_size"] = chunk_size
        node.excluded_embed_metadata_keys = ["chunk_size"]
        node.excluded_llm_metadata_keys = [ "chunk_size"]

        nodes_list.append(nodes)

        vector_index = VectorStoreIndex(nodes)
    vector_indices.append(vector_index)

권

8 comments

권권씨 😮💨

WWhiteFang_Jr

Default is ADA 002:
https://github.com/run-llama/llama_index/blob/70c16530627907b2b71594b45201c1edcbf410f8/llama-index-integrations/embeddings/llama-index-embeddings-openai/llama_index/embeddings/openai/base.py#L287

권권씨 😮💨

Thx ❤️

권권씨 😮💨

can you help this too?

Is it possible to compare and evaluate performance between retrievers?

For example, let's say I implemented an ensemble retriever and a BM25 retriever. And can you compare the performance of the two?

What I'm curious about is whether it is possible to compare and evaluate implemented retrievers.

WWhiteFang_Jr

You can compare the response in two ways in my opinion:

You can compare the retrieved nodes with the query and check relevancy or if you have some set of nodes that should have been returned you can compare it with that.
You can use GPT4 to check if the retrieved nodes are as per what query was asked or not.

권권씨 😮💨

Is it true that comparing and evaluating implemented retrievers is a module that is not yet supported by llamaindex?

Like response comparison evaluation

WWhiteFang_Jr

Not entirely sure but yea looks like comparing retrievals is not implemented yet.
For single retrieval evaluation, you can check this out: https://docs.llamaindex.ai/en/stable/examples/evaluation/retrieval/retriever_eval/

권권씨 😮💨

What is a single retriever? Can an ensemble retriever do it too?

Add a reply

Find answers from the community

When using openai gpt4 turbo for llm in