Find answers from the community

Updated 2 months ago

Hi, maybe someone knows:

Hi, maybe someone knows:

i want to use a local embedding llm with chromaDb. running localy, donwloaded from hugging face and fine tunned (1024 dimensions)
Plain Text
collection = client.get_or_create_collection(vhector_id)
    vector_store = ChromaVectorStore(chroma_collection=collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)  # type: ignore
    # service_context = ServiceContext.from_defaults(embed_model="local:llm/embeddings")
    index = VectorStoreIndex(
        nodes,
        storage_context=storage_context,
        # service_context=service_context,
    )
`
when creating index, it takes forever, are there some settings to put ?
W
L
V
4 comments
You could try using async mode. It will reduce some time for your case.

Plain Text
index = VectorStoreIndex(
        nodes,
        storage_context=storage_context,
        service_context=service_context,
        use_async=True
    )
use_async sadly won't help with local embeddings, only with API based embeddings

For local embeddings, your best bet is to make sure you are using a GPU, or to increase the batch size

HuggingFaceEmbeddings(..., embed_batch_size=20)
ok, i don't have to set the model dimensions omewhere ?
chroma should automatically figure out the dimensions
Add a reply
Sign up and join the conversation on Discord