Find answers from the community

Updated 2 months ago

Maybe I need to change embedding model?

I am using llama2 in the code but why do I still need OpenAI API key

My .env file has this structure:
Plain Text
OPENAI_API_KEY=<key>
REPLICATE_API_KEY=<key>
B
L
46 comments
I have set the embed_model="local" as well
You are creating the service context, but you are not passing it anywhere
I am passing it in the query engine
Plain Text
            query_engine = self.index.as_query_engine(service_context=self.ctx, streaming=True)
That doesn't seem to work

Try this

Plain Text
self.index = VectorStoreIndex.from_documents(documents, service_context=self.ctx)
# or
self.index = load_index_from_storage(storage_context, service_context=self.ctx)
Should I remove it from query engine?
yea remove it for now πŸ€”
Tried removing it
still the same issue
requires OPEN_AI key
even with embed_model="local" ?
It worked now
I had to delete the storage
now its working
@Logan M I have to delete storage everytime I want to make it work
otherwise it doesn't work and keeps asking openai key
do you know why that is?
You added this line?
self.index = load_index_from_storage(storage_context, service_context=self.ctx) ?
adding it now
so I had to add context in all 3 places to make it work
let me try again
@Logan M still the same issue
first run without storage works
but it doesn't work in the second run
Can you send your current code again?
in the loading path, self.ctx is none
Plain Text
def _initialize_index(self):
    self.ctx = ServiceContext.from_defaults(llm=llm, embed_model="local")
    if not os.path.exists(self.persist_dir):
        documents = SimpleDirectoryReader("data").load_data()
        LLAMA_13B_V2_CHAT = "a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5"
        llm = Replicate(
            model=LLAMA_13B_V2_CHAT,
            temperature=0.01,
            context_window=4096,
            completion_to_prompt=self.custom_completion_to_prompt,
            messages_to_prompt=messages_to_prompt,
        )
        self.index = VectorStoreIndex.from_documents(documents, service_context=self.ctx)
        self.index.storage_context.persist(persist_dir=self.persist_dir)
    else:
        storage_context = StorageContext.from_defaults(persist_dir=self.persist_dir)
        self.index = load_index_from_storage(storage_context, service_context=self.ctx)
that would work
initializes the ctx regardless of the if condition
ohhh no I am a dumbass
ty let me try
also @Logan M what does it mean by embed_model="local"
conceptually i mean
It's shorthand for embed_model=HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Add a reply
Sign up and join the conversation on Discord