why would someone do this index struct

I have no idea. Sounds like a recipe for bugs if you ask me

okay thats what i thought wanted to double check thank you!

Sorry I was looking more into it the reason I wanted to do it is because the index of a condense chat engine expected a vectorstore index rather than a base index however when i try to form the index as above the chat outputs very wrong answers

sorry to be a little more clear "index=self.index," this gives me the type error "expects VectorStoreIndex" however when i do what I did above i do not think the chat engine is querying the right index

Hmm, that error doesn't seem possible? The CondenseQuestionChatEngine can take any query engine

Note that the expected type is BaseQueryEngine

ah sorry i did not mean error i meant

but thank you again!

what's the line above that? What class are you instansiating? 👀

oh sorry for the late response it is a custom retriever class to retrieve embeddings rather than strings

Ah I see.

That's just a pylint error, so it's not a real error. It's just because the custom retriever expects a vector index.

You can do an assert isinstance() check for the type, or just cast the type to fix the "error"

sorry im a little confused how would i cast the type to make it a VectorIndex

Also is the new default llm llama-cpp i just tried running it and it is giving me a very different output

Well it's a completely different llm, so it will be wildly different 😅 it's just something to quickly unblock people.

You could definitely try downloading and usinf a different llama cpp model.

It's just a python typing thing

It's just cosmetic though

hahah yeah sorry it was more unexpected because of all the logging and thank you so much for your help, I am going to try to load a model directly from hugging face and use the quantized version so I may have some more questions in a little while

Sorry I am back and have a few questions: if i dont specify an open ai key but specify the model like self.llm = llm if llm else load_llama_v2_8bit() why is the logging for the cpp still coming up?

Also if i do then specify the api key it seems to load in the model and everything but my account is still being charged

Where/how are you specifying the model?

im creating a bot class and specifying the llm inside the class like "llm if llm else load_llama_v2_8bit() "and then setting up the service context like "service_context = ServiceContext.from_defaults(
llm=self.llm,
embed_model=self.embed_model
)
condense_engine = CondenseQuestionChatEngine.from_defaults(
service_context=service_context,"

Ooo the chat engine doesn't actually use the service context there lol

You can pass in the llm directly as a kwarg

like this sorry "condense_engine = CondenseQuestionChatEngine.from_defaults(
llm=self.llm,
query_engine=RetrieverQueryEngine(
retriever=FAISSRetriever(
index=cast(VectorStoreIndex, self.index),
embed_model=self.embed_model
)
)


)"

sorry im also a little confused because llama cpp is still popping up but i do nto want to use it

sorry to spam but i have also tried with a context chat engine " def _get_context_engine(self) -> BaseChatEngine:
"""initialize a llama-index context engine"""

service_context = ServiceContext.from_defaults(
llm=self.llm,
embed_model=self.embed_model
)

context_engine = ContextChatEngine.from_defaults(
service_context=service_context,
retriever=FAISSRetriever(
index=cast(VectorStoreIndex, self.index),
embed_model=self.embed_model
)
)
return context_engine" and this one does not have the llama.cpp logging throughout the chat however it does in the beginning

Hmmm the only thing I can think is that llama cpp is being activated in the index?

Hard to say without seeing the whole flow I guess

But If it wasn't llama cpp, it would be key errors about openai lolb

oh lol would it be possible not to have an api key and also not have the llama cpp log or the openai key errors

or specify an api key but use another llm so that openai would not be used

also is that the right way to specify an llm in the condense chat engine

if it is okay with you i can also pm the whole flow

Maybe pm the whole flow, just to double check lol shouldn't be too hard