```py

You have only defined the llm part in the service_context . It requires embedding model source as well. Since it is not defined it is going to OpenAI for it.

Just add the following and it should work!!

Plain Text

service_context = ServiceContext.from_defaults(llm=llm, embed_model="local")

It will use the default opensource model for embedding this way.

For more: https://docs.llamaindex.ai/en/stable/module_guides/models/embeddings.html#local-embedding-models

Okay. If i define it it gives me a "Empty Response", Is that when it doesnt know wjat to say?

If the response returns empty response, Can you check if the source nodes contain anything.

You can check via print(response.source_nodes)

If it is coming empty then it means the embedding model is unable to retrieve from the vectors

AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
Empty Response
[]

It gives me that

So yea, its empty

So meaning the DB is borked? Or does it just not have Info abt what i asked?

How did you created the vector store, IN the first place? Have you fed any data in the vector store yet?

Yes. It does have data in it. Abt 10 PDF's about Agrar stuff. And its fully build. Its a ChromaDB

But no matter what i ask. I get no response

Lol, I was going to say try with some other queries 😅

Gonna try sth that is defo in the PDF

still empty response

Lets try it in the building process

Can you try creating it from the start once again: https://docs.llamaindex.ai/en/stable/examples/vector_stores/ChromaIndexDemo.html#creating-a-chroma-index ?

Could be some issue in the vectors maybe

doing that rn actually

Printing response
DerProzess„Düngen“beinhaltetmehrereSubprozesse,dieengverzahntineinandergreifen:
•„Anbauplanen“istdieVerknüpfungeinerKulturartmiteinerBewirtschaftungs-
einheitineinembestimmtenZeitraum.DieKulturart,sowiedieMengeundArt
desErnteproduktesbestimmtimWesentlichendieAusbringungvonNährstoff-
mengen.
•„Bodenuntersuchungeingeben“beinhaltetdenTransfervon(teilflächenspezifischen)BodenuntersuchungsergebnissenodervonLandesbehördenzurVerfügung
gestellteNmin-GehalteindieAnwendung.
•„Düngebedarfermitteln“umfasstdieBerechnungderNährstoffmengeeinesErn-
teproduktes,diedenNährstoffbedarfnachAbzugsonstigerverfügbarerNähr-
stoffmengenundunterBerücksichtigungderteilflächenspezifischenNährstoffver-
sorgungdesBodensabdeckt.
•„Düngebedarfdecken“isteinteilflächenspezifischerOptimierungsansatzzur
DeckungdesDüngebedarfs(sieheAbb.1).
•„Düngungplanen“liefertalsErgebniseineernteprodukt-undteilflächenspezifischePlanungvonDüngemaßnahmeninkl.Applikationskarte.
InderAnwendungmüssendieProzessevomUserdurchlaufenundggf.umfehlende Informationenangereichertwerden.InAbb.1istderProzess„Düngebedarfdecken“
exemplarischdargestellt.DerUser(„Besitzer“)wirddabeivonderAnwendung(„Sys-
tem“)durchdenProzessbegleitet.WennalleInformationenausdenvorherigenProzes-
senkorrekterfasstunddieGeschäftsregelnkorrektformuliertsind,somussderAnwen-
derlediglicheinErgebnisbestätigen.
Abb.1:AusschnittausBPMNDiagrammfür„Düngebedarfdecken“
taym@tays-MacBook-Pro AiAssistant %

Oh hell

I mean it is right

Just

it has no spaces between the words

Awesome !! now the response is finally coming
Yeah, You could try providing instruction to the model for how it has to generate and ask again.

Okay so, when i save the ChromaDB it doesnt work. It doesnt respond

So somethig is defo borked

Can you check, If the new file got stored in your chromaDB

Attachment

But i just tried sth

Gives me a File Exists error

Attachment

Can you try providing just ./chroma_db in the path, as this much is only mentioned in the docs
https://docs.llamaindex.ai/en/stable/examples/vector_stores/ChromaIndexDemo.html#basic-example-including-saving-to-disk

I did that and there i gave me empty response

I think you've set the context window and amx_new_tokens waaayy too big 😅

What model is this based off of?

i would try max_new_tokens=512 and context_window=3900 as a safe starting point

Its running on Em German 13b

By TheBloke I think

With the context window it hissed at me when I had it at 3900 that it can't go into the negatives

lol wut

hmm

yee

Okay i tried so many things. Nothing really works. Could it be a issue with my hardware? idfk

sorry for the ping

May have found a solution

Just sadly takes long

My solution did in fact not work.
But in the file where it was building it so it had it already.
I got this response lol:

Plain Text

DerProzess„Düngen“beinhaltetmehrereSubprozesse,dieengverzahntineinandergreifen:
•„Anbauplanen“istdieVerknüpfungeinerKulturartmiteinerBewirtschaftungs-
einheitineinembestimmtenZeitraum.DieKulturart,sowiedieMengeundArt
desErnteproduktesbestimmtimWesentlichendieAusbringungvonNährstoff-
mengen.
•„Bodenuntersuchungeingeben“beinhaltetdenTransfervon(teilflächenspezifischen)BodenuntersuchungsergebnissenodervonLandesbehördenzurVerfügung
gestellteNmin-GehalteindieAnwendung.
•„Düngebedarfermitteln“umfasstdieBerechnungderNährstoffmengeeinesErn-
teproduktes,diedenNährstoffbedarfnachAbzugsonstigerverfügbarerNähr-
stoffmengenundunterBerücksichtigungderteilflächenspezifischenNährstoffver-
sorgungdesBodensabdeckt.
•„Düngebedarfdecken“isteinteilflächenspezifischerOptimierungsansatzzur
DeckungdesDüngebedarfs(sieheAbb.1).
•„Düngungplanen“liefertalsErgebniseineernteprodukt-undteilflächenspezifischePlanungvonDüngemaßnahmeninkl.Applikationskarte.
InderAnwendungmüssendieProzessevomUserdurchlaufenundggf.umfehlende Informationenangereichertwerden.InAbb.1istderProzess„Düngebedarfdecken“
exemplarischdargestellt.DerUser(„Besitzer“)wirddabeivonderAnwendung(„Sys-
tem“)durchdenProzessbegleitet.WennalleInformationenausdenvorherigenProzes-
senkorrekterfasstunddieGeschäftsregelnkorrektformuliertsind,somussderAnwen-
derlediglicheinErgebnisbestätigen.
Abb.1:AusschnittausBPMNDiagrammfür„Düngebedarfdecken“

So no Spaces. Still didnt get it to save and make it accessible that i dont have to rebuild the DB every time i ask it sth

name='quickstart' id=UUID('0a6d6743-bbef-4f98-9897-6ff994a84a1b') metadata=None

Hmm thats the collection

The collection UUID is diff

Building DB 
name='quickstart' id=UUID('560f6a12-e64b-467b-9a5f-3b4393fa160f') metadata=None

Getting DB
name='quickstart' id=UUID('0a6d6743-bbef-4f98-9897-6ff994a84a1b') metadata=None

ngl I'm way out of the loop now -- what are you doing exactly?

I copied the code entirely as the guide tells me to. Still didn't work. But I tried comparing the collection UUID's. From the file where I saved the chromadb. And then the file where I then tried getting the previously saved chromadb

Which code?

In the Llama Index Docs

The sample code

For chroma?

Loading a previous chromadb is ez pz

Plain Text

# save to disk
db = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = db.get_or_create_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
service_context = ServiceContext.from_defaults(embed_model=embed_model)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, service_context=service_context
)

# load from disk
db2 = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = db2.get_or_create_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
index = VectorStoreIndex.from_vector_store(
    vector_store,
    service_context=service_context,
)

this works for me