every instance of a chat engine is instantiated with its own memory
You can also pass in your own chat_history as a kwarg to every
chat() call to override the built in memory
For example
chat_history = [ChatMessage(role="system", content="talk like a pirate")]
chat_engine.chat("Hello!", chat_history=chat_history)
# get latest chat history
latest_history = chat_engine.chat_historyYou can access any sources using response.sources
If your chat engine is built using an index, response.source_nodes also works
When printing source_nodes i get an empty array [], and when printing the response.sources i get the system prompt i gave the chat_engine.
Does this means that its not loading data from the vector_store and getting the info from the pretrained model (gpt-3.5-turbo) ?
hmmm I wonder how the system prompt got into the sources lol
If the resonse_nodes are empty, this means it isn't calling an vector index yea 🤔 What kind of chat_engine are you using?
This is my code
chat_engine = index.as_chat_engine(
chat_mode="context",
memory=None,
system_prompt="""\
Actuaras como un escribano de latinoamerica\
No deberas citar link externos, solo titulos de los archivos de los que has sido entrenado
""",
verbose=True,
)
response = chat_engine.stream_chat(input_text)
stream_response = response.response_gen
for text in stream_response:
yield text
print (response.sources)
print(response.source_nodes)
chat_engine.reset()only the chat_engine part
ohhhh ok this makes more sense
The response.sources in this case is the system message, because the chat engine retrieves context from the index and throws it into the system message
response.source_nodes should be populated though, unless your index is empty?
(just reading the code right now)
This is how i load my index:
num_outputs = 1024
llm = OpenAI(
temperature=0.1,
model="gpt-3.5-turbo-16k",
max_tokens=num_outputs,
streaming=True,
)
db2 = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = db2.get_or_create_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
service_context = ServiceContext.from_defaults(llm=llm)
set_global_service_context(service_context)
index = VectorStoreIndex.from_vector_store(
vector_store=vector_store,
storage_context=storage_context,
service_context=service_context,
)I have the path correct, and i have about 1.9gb in that collection
if you query with a regular query engine, does it work?
With this code:
query_engine = index.as_query_engine()
response = query_engine.query("Como es tu nombre")
print(response.response)
print("Source Nodes")
print(response.source_nodes)I got :
Using the index mentioned above
ok! so we narrowed down the source of the problem 💪
it's either not connecting back to chroma properly, or the chroma index is empty 🤔
Maybe try this instead when loading the index?
db2 = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = db2.get_or_create_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
service_context = ServiceContext.from_defaults(llm=llm)
set_global_service_context(service_context)
index = VectorStoreIndex.from_vector_store(
vector_store,
service_context=service_context,
)So does this means that it has been loading the answers from gpt-3.5 all this time?
Mmm nop i get the same response:
maybe use get_collection instead of get_or_create_collection ?
I wonder if chroma has a bug
This didn't change anything hahah
How can i know if there is data on my chroma vector store, or if its loading correctly. Apart from seeing the file size
I have no idea hahaha I'm assuming the chroma api docs might detail some functions we could use
Imma run the notebook we have as a sanity check though
Ok, my sanity check worked, which is extra confusing for you I suppose haha
Yep, still the same result
Can you try re-creating the initial index? Maybe something went wrong?
Yes sure, i created it on another machine and transfered all the files to my computer. But i will do it again
Can it be something related to the embde_model?
This is very similar to the example on their documentation:
# create client and a new collection
chroma_client = chromadb.EphemeralClient()
chroma_collection = chroma_client.create_collection("quickstart")
embed_model = LangchainEmbedding(
HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2")
)
# load documents
documents = SimpleDirectoryReader("docs").load_data()
# save to disk
db = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = db.get_or_create_collection("quickstart")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
service_context = ServiceContext.from_defaults(embed_model=embed_model)
index = VectorStoreIndex.from_documents(
documents, storage_context=storage_context, service_context=service_context
)Hahah. I am losing my mind
OK check #1
documents = SimpleDirectoryReader(...).load_data
If you print the documents, did it actually load anything?
I have a lot of files, but yeah if i remove some i get it printed
I went to the PC that trained the data, and i am printing the sources and i get them right
But when trying to do it from my PC
Can it be because i transfered files from the PC to mine?
Maybe? 🤔 I'm not 100% sure with chroma
That's extremely weird though h
I will keep you posted!! Thanks for the help @Logan M
