Hi, maybe someone knows:

VVH-TLS

Hi, maybe someone knows:

i want to use a local embedding llm with chromaDb. running localy, donwloaded from hugging face and fine tunned (1024 dimensions)

Plain Text

collection = client.get_or_create_collection(vhector_id)
    vector_store = ChromaVectorStore(chroma_collection=collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)  # type: ignore
    # service_context = ServiceContext.from_defaults(embed_model="local:llm/embeddings")
    index = VectorStoreIndex(
        nodes,
        storage_context=storage_context,
        # service_context=service_context,
    )

`
when creating index, it takes forever, are there some settings to put ?

4 comments

WWhiteFang_Jr

You could try using async mode. It will reduce some time for your case.

Plain Text

index = VectorStoreIndex(
        nodes,
        storage_context=storage_context,
        service_context=service_context,
        use_async=True
    )

LLogan M

use_async sadly won't help with local embeddings, only with API based embeddings

For local embeddings, your best bet is to make sure you are using a GPU, or to increase the batch size

HuggingFaceEmbeddings(..., embed_batch_size=20)

VVH-TLS

ok, i don't have to set the model dimensions omewhere ?

LLogan M

chroma should automatically figure out the dimensions

Add a reply

Find answers from the community

Hi, maybe someone knows: